See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/335700462

Discriminatory Artificial Intelligence in Organizations: Causes and Prevention

Methods

Article · May 2019

CITATIONS READS

0 230

1 author:

Stephanie Kelley
Queen's University
5 PUBLICATIONS   0 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

A Code of Conduct for Ethical AI in Financial Services View project

(Anti-Discrimination) Law, AI, and Gender Bias View project

All content following this page was uploaded by Stephanie Kelley on 09 September 2019.

The user has requested enhancement of the downloaded file.

Discriminatory Artificial Intelligence in Organizations: Causes and Prevention Methods

Stephanie Kelley, stephanie.kelley@queensu.ca

Smith School of Business, Queen’s University, Kingston, Canada

Abstract

Many organizations have proliferated bias and discrimination through their use of artificial

intelligence (AI), despite adhering to anti-discrimination laws. I investigate the reasons why anti-

discrimination laws in North America and Europe are ineffective when applied to AI, arguing

they allow for easy justification of discrimination, make it difficult to prove intent, and ignore

forms of statistical bias which are unique to AI models. I define statistical biases and other AI

terms for managers using non-technical language and a series of real-world examples. In

addition, I propose that the existing laws could artificially inflate perceptions of distributive

justice which could paradoxically dissuade organizations from attempting to reduce AI

discrimination. I then recommend specific actions managers can take to prevent AI bias and

discrimination in their organizations.

Keywords: anti-discrimination law, artificial intelligence, AI discrimination, AI bias, statistical

bias, artificial intelligence ethics, AI ethics, IT ethics, distributive justice, procedural justice

1
Introduction

We are living through a democratization of artificial intelligence (AI), a time where

organizations and individuals are gaining access to an unprecedented number of AI technologies

from deep neural nets (Brynjolfsson et al. 2018) to virtual assistants (Wilson and Daugherty

2018). In organizations, the increased access has lead to a drastic rise in the use of AI to inform

business decisions, which has in turn lead to many business wins (Davenport and Katyal 2018).

Yet somehow during its rapid adoption, many organizations have forgotten that artificial

intelligence, like any technology, has ethical implications (Khalil 1993; Martin 2018).

One of the most pressing ethical issues is artificial intelligence’s ability to proliferate bias and

discrimination (Campolo et al. 2017; Koene 2017), particularly against individuals from

protected classes, such as those of minority race, gender, and national origin (Andreeva et al.

2004). Bias and discrimination can be proliferated by artificial intelligence because there is a gap

between the existing anti-discrimination laws and the technology’s new capabilities (Barocas and

Selbst 2016; Zarksy 2014). This gap has been observed before and is referred to as cultural lag:

material culture such as physical equipment or production procedures, in this case AI, advances

more rapidly than non-material culture including ethics, philosophy, belief systems, values and

laws (Marshall 1999). Cultural lag develops because the process of establishing laws and policies

is challenging and therefore falls behind the rapid pace of technological developments (Wellman

and Rajan 2017). Unfortunately for organizations, the cultural lag means that adhering to

existing laws is not enough to prevent discrimination when using artificial intelligence for

business decisions.

Regardless of the cultural lag, organizations using AI are still held responsible and accountable

for the technology’s outcomes (Martin 2018). This reality has been felt by many organizations

2
who have operated within the confines of the law but have been deemed at fault for the

discriminatory use of AI, including Amazon, who created a gender-biased hiring algorithm

(Dastin 2018); Microsoft, IBM, and Chinese company Megvii, who developed skin-colour-

biased facial recognition algorithms (Cossins 2018); and LinkedIn, with their gender-biased

search algorithm (Day 2016). These organizations were all found to be operating within the

confines of the law, and were believed to have made the discriminatory AI systems

inadvertently, without explicit intent to proliferate bias, yet were still held responsible and

accountable by the public for the discrimination (Cossins 2018; Dastin 2018; Day 2016). These

examples highlight real cultural lag, an obvious gap between how AI can be used by an

organization and how AI should be used (Davenport and Katyal 2018).

In this paper I discuss why the current anti-discrimination laws in North America and Europe are

ineffective at preventing discrimination when applied to AI-based decision-making processes, a

multidisciplinary issue currently under investigation by researchers across management (Martin

2018), computer science (Kirkpatrick 2016), law (Barocas and Selbst 2016; Kastellec 2010;

Zarksy 2014), and statistics (Bower et al. 2018), as well as by industry practitioners (Campolo et

al. 2017; Murgia 2019). I argue that the existing anti-discrimination laws are ineffective when

applied to AI because they allow for easy justification of discrimination (Barocas and Selbst

2016), make it difficult to prove discriminatory intent (Zarksy 2014), and ignore statistical

biases, a set of mathematical properties unique to data-driven processes which can lead to

discrimination in new and confounding ways (Barocas et al. 2018). I then propose that in

addition to being ineffective, the existing anti-discrimination laws could also dissuade

organizations from further attempting to prevent discrimination, as they could lead to an

overestimation of the fairness of the decision outcomes. Finally, I present several methods which

3
organizations could implement to fill the gaps in the existing anti-discrimination laws to prevent

the proliferation of bias and discrimination from AI.

Justification, Intent, Inconsistencies and Statistical Bias: Why the Law Can’t Prevent

Discriminatory AI

Justification

Anti-discrimination laws exist in principle to promote the fair and, or equal treatment of all

individuals, and do so by prohibiting discrimination against protected classes (Marie 2016).

What constitutes discrimination against a protected class varies across national law, but generally

refers to any situation where an individual from a group distinguished by attributes protected by

law are disadvantaged (Barocas and Selbst 2016). Variances across legal jurisdictions ultimately

impacts how AI is regulated in each country, but there are many consistencies in the cultural lag

of the American, European Union, and Canadian anti-discrimination laws that have led to similar

discriminatory AI issues (Barocas and Selbst 2016; Canadian Human Rights Act 2017; European

Union Agency for Fundamental Rights and Council of Europe 2018).

The laws of these countries prohibit discrimination based a set of protected characteristics, such

as race, gender, sex, sexual orientation, disability, and religion or belief, but they all allow

businesses to justify the use protected attributes if they can prove some legitimate business

rationale (Barocas and Selbst 2016; Čerka et al. 2015; Zarksy 2014). What qualifies as legitimate

rationale for the use of a protected attribute varies across countries, but as an example in Canada,

the Canadian Human Rights Act has an exception which allows organizations to use protected

attributes if they can provide a bona fide justification, showing that excluding the characteristic

4
“would impose undue hardship on the person [or organization] who would have to accommodate

those needs, considering health, safety and cost” (Canadian Human Rights Act 2017).

In practice, organizations are usually able to provide legitimate rationale for the use of a

protected attribute in an AI model, regardless of the discriminatory impact, due the fundamental

mathematical properties of AI which rely on correlations between variables to predict a decision

outcome (Barocas and Selbst 2016; Taylor 2011). For example, a credit lender may want to

create an AI model to predict applicant default, and they select zip codes as a predictive variable

since they believe them to be correlated with borrower default. But zip codes are also known to

be highly correlated with race (Taylor 2011), which is a protected attribute under most anti-

discrimination laws; the correlation means the AI model outcome could be racially

discriminatory, even though race data isn’t explicitly included in the model. As long as zip codes

have some explanatory power in predicting borrower default, removing the variable will

decrease the AI model’s predictive power, leading to a less accurate model (Chan and Seow

2013). Less accurate models are usually less profitable (Blöchlinger and Leippold 2006), in the

credit lending example lower model accuracy means the lender would accept a higher number of

“risky” applicants, ultimately leading to higher financial loss. The clear trade off between

accuracy and profitability means organizations can often justify keeping variables highly

correlated with protected attributes, or even the protected attributes themselves (if the law

doesn’t explicitly prohibit it) in the name of profit, regardless of the discriminatory impact of the

decision outcome.

Intent

Some anti-discrimination laws, particularly those in the United States also require proof of

discriminatory intent in order for an organization to be found liable in court (Barocas and Selbst

5
2016; Zarksy 2014). However, AI models make it difficult to prove intent due to the correlations

between variables, and opaque or highly complex algorithms (Burrell 2016; Zarsky 2016). A

prejudice AI designer could select certain variables that are highly correlated with protected

attributes in order to mask their intent to discriminate, per the credit lending example, they could

explicitly select zip codes knowing that they are correlated with race. Alternatively, they could

choose to use an opaque or highly complex algorithm to intentionally conceal the discrimination

in the AI model (Burrell 2016). Unfortunately, these forms of statistical bias, which will be

discussed in more detail below, could also occur without any discriminatory intent; the data

scientist may have selected zip codes, or a complex algorithm simply because they improve the

model’s accuracy, making it difficult to cite the use of certain variables or models as proof of

discriminatory intent in a court of law (Zarksy 2014). If intent cannot be shown, then the

protected attribute can continue to be used with the AI model.

Inconsistencies

Even more confusing for organizations and the courts is that researchers have uncovered some

instances where paradoxically the use of a protected attribute in an AI model actually leads to

more beneficial outcomes for individuals in the protected class (Andreeva and Matuszyk 2018;

Chan and Seow 2013). For example, Andreeva and Matuszyk (2018) investigated the use of

gender as a predictive variable in AI models for credit scoring. The authors created two credit

scoring models, one with gender, which is prohibited by most anti-discrimination laws, and one

without gender, in line with the relevant laws (Andreeva and Matuszyk 2018). They found that

the model that included gender, which “broke the law,” was more beneficial for women as it

provided them larger amounts of credit than the model that adhered to the law and excluded

6
gender from the model (Andreeva and Matuszyk 2018). This example further highlights the

cultural lag of the existing anti-discrimination laws when applied to artificial intelligence.

Statistical Biases

The existing anti-discrimination laws also ignore the reality that AI-based discrimination issues

tend not to proliferate the same types of discrimination that humans do (Kirkpatrick 2016; Martin

2018; Mittelstadt et al. 2016; Zarksy 2014). The current laws were created before these statistical

biases were well understood, and therefore don’t address them effectively (Floridi et al. 2017).

Many of these statistical biases have and continue to be explored by statisticians (Bower et al.

2018) and computer scientists (Kirkpatrick 2016), but have yet to be discussed in the

management ethics literature. To rectify this, I discuss the types of statistical bias, in context of

the AI-based decision-making process (see Figure 1) using a series of real-world examples to

help educate managers on the unique ways AI can proliferate bias (Custers et al. 2013)

Statistical Bias in Data Collection

Sample Selection Bias

Missing, partial, outdated, and nonrepresentative datasets can all lead to discriminatory decision

outcomes (Banasik et al. 2003; Barocas and Selbst 2016; Custers et al. 2013; Martin 2015;

7
Williams et al. 2018). AI models rely on training data, a sample of information from the larger

population the model is being applied to, that is then used by the model (also referred to as an

algorithm) to arrive at a decision outcome (Martin 2018). The training data is what the AI model

learns is the “truth”, and it builds all predictions and decisions based on that data (Williams et al.

2018). If the “truth” is incomplete in some way, the output will be less accurate compared to if

the dataset was complete (Custers et al. 2013). This also means AI models are more accurate at

predicting decision outcomes for observations in the training data, or observations that closely

resemble the training data, compared to observations that are unique (Banasik et al. 2003). The

World Economic Forum has acknowledged that protected groups may generate less data than

others because of historical inequalities, meaning protected groups may be more likely to be left

out of datasets, and therefore more likely to receive inaccurate predictions from AI models

(World Economic Forum Global Future Council on Human Rights 2016-18 2018). Whilst

missing, partial, and outdated datasets may be easy to diagnose due to their obvious errors,

nonrepresentative datasets, which do not accurately capture the entire population of individuals

impacted by the AI model, may be more challenging to recognize, making them a greater

challenge for anti-discrimination efforts (Williams et al. 2018).

Biased Data Manipulation

The way in which training data is manipulated (edited, organized, changed, etc) can also

proliferate discrimination, especially if past historical biases are not corrected for before feeding

the training data into the algorithm (Barocas and Selbst 2016; Executive Office of the President

National Science and Technology Council Committee on Technology 2016). This is especially

important when using historical datasets as they may have recorded prejudicial actions or

discriminatory beliefs into the decision outcomes, even though the decision makers today don’t

8
carry those same beliefs (Custers et al. 2013). Continuing with the credit lending outcome; if a

bank has historically not provided credit to a protected group for prejudicial reasons, let’s say

Indigenous men, then the AI model would learn from the historical data to reject applicants who

are Indigenous men, even if there is no financial reason to do so. Current human biases can also

be learned by the algorithm through a feedback loop, even if it was originally trained on

unbiased data (Barocas et al. 2018). This can occur if an AI models is used to support human

decision-making processes; a human uses the output from the AI model along with other non-AI

methods to arrive upon a final decision outcome, which is then fed back into the training data

(Custers et al. 2013). If the human decision maker alters the decision outcome in a

discriminatory way, for example by denying credit to all individuals in a given protected class,

even though the AI model predicted they would be repay their credit, the AI model will learn this

discriminatory behaviour from the human feedback loop and codify it for future decisions.

Biased Feature Selection

Features are the data attributes or variables selected for use in an AI model, with feature

selection referring to the process of choosing those attributes or variables (Barocas and Selbst

2016; Custers et al. 2013). Features can include traditional variables such as economic

information, customer attributes, or sales data, but can also include mathematical combinations

or transformations of variables. Discrimination can occur because the features selected are not

granular enough for the model to accurately differentiate between data observations to arrive

upon an accurate, and ultimately fair decision outcome (Barocas and Selbst 2016; Bart 2017). An

AI model for credit scoring might include zip code as a feature, which as discussed, is highly

correlated with race. If no other strong predictors are included, the model would rely exclusively

on the zip codes, taking the average creditworthiness of each geographic area and assigning that

9
to everyone from the neighbourhood, instead of a developing a more accurate estimate of default

for each individual using more predictive variables. As race is correlated with zip codes, the

credit granting decisions would be discriminating based on applicant race. Biased feature

selection can also lead to a sort of “rational racism,” a term coined by Barocas and Selbst (2016),

which describes a situation where a protected data attribute is used in an AI model because it is

believed to be correlated with some other meaningful attribute which for one reason or another

can’t be gathered. For example, until recently, Facebook allowed advertisers to use detailed

demographic information to target ads, under the belief that sharing the protected data attributes

would create highly targeted ads that would benefit the consumer, a “benefit” not possible

without using race or other protected attribute data (Dopp and Westbrook 2019). This type of

discrimination can occur where there are no anti-discrimination laws in place that explicitly

condone the use of protected attributes in AI models.

Biased Decision Outcome Variables

In order to use most AI models, particularly classification models, an organization must pick a

decision outcome variable for the model to calculate, one that is available in the dataset (Barocas

and Selbst 2016; Custers et al. 2013). Often the exact decision outcome of interest is not

available so organizations must in lieu choose a variable that they believe is closely related to

their decision outcome of interest, which can lead to discriminatory outcomes (Barocas and

Selbst 2016). In the credit lending example, a creditor cannot know with certainty the future

repayment behaviour of applicants, so they use past borrower behaviour to predict the future

creditworthiness of similar applicants. The lender might decide the any borrower who defaulted

in the past is not worthy of credit, so they train their AI model to deny credit to any applicant that

looks like a borrower who has defaulted. The potential for bias arises if the decision outcome

10
variable, in this case past default, is not a perfect predictor of future default; and of course,

without a crystal ball, no decision outcome variable is perfect. Potential bias is then realized if

the decision outcome variable is correlated with some protected attribute that is intrinsically

linked to the decision outcome variable. For example, past default rates may be higher for

persons with disabilities due to historical discriminatory hiring practices in the market, which

lead to fewer persons with disabilities being employed and therefore historically less able to pay

their credit lender. If the creditor simply relies on past default rates, without understanding the

nuances of selecting that variable as the decision outcome, they can potentially proliferate further

bias and discrimination.

Statistical Bias in Model Development

Proxy Variable Correlation to Protected Attributes

Proxy variables are variables which are highly correlated with other variables of interest (Zhang

2018). Their use can lead to discriminatory decision outcomes through either malicious or

inadvertent intent (Barocas and Selbst 2016; Custers et al. 2013). Individuals involved in the AI

modelling process could use proxy variables in place of protected attributes to hide their

discriminatory intent (i.e. zip codes in place of race), but they may also do so inadvertently,

included variables highly correlated with protected attributes in the model without realizing the

correlation (Barocas and Selbst 2016). In practice, this occurred at Amazon when they created a

biased AI hiring model that gathered gendered terms from applicant resumes, such as “women”,

“her”, and “girls” from phrases like “women’s soccer team,” and rejected any applicant found to

have these words in their resumes (Dastin 2018). Amazon apparently did this inadvertently as

they had trained the algorithm to provide high scores for applicants who appeared similar to their

current employees, most of whom are men, so the algorithm learned to penalize terms associated

11
with being a woman, which ultimately lead to a discriminatory hiring process (Dastin 2018).

Regardless of the intent, proxy variables, like gendered words in place of actual gender can lead

to discriminatory decision outcomes, and are especially dangerous in large datasets and

unsupervised machine learning models that rely heavily on correlations, often outside the explicit

knowledge of the modellers, to arrive at a decision outcome (Barocas and Selbst 2016).

However, as mentioned in the discussion of intent earlier, most AI models rely on variable

correlations to generate a decision outcome, so simply removing all variables correlated with

protected attributes is not a feasible solution to prevent discrimination, leaving organizations

without a clear resolution (Chan and Seow 2013; Custers et al. 2013).

Statistical Bias in Monitoring and Review

Inadvertent Bias Due to Unforeseen External Factors

A nascent line of research, has uncovered that even when an organization takes care to prevent

discrimination in the development of the AI model, the decision outcome can be influenced by

external, apparently neutral factors, that lead to “accidental” discrimination (Custers et al. 2013;

Lambrecht and Tucker 2019). Lambrecht and Tucker (2019) show that display ads for STEM

(science, technology, engineering, and math) jobs which they thought were being sent to equal

proportions of men and women were in fact being sent disproportionately to men, proliferating

the existing bias that women are underrepresented in STEM careers. What was alarming was that

the discriminatory outcome was not linked to a historical gender bias, it was instead linked to the

economic valuation of each gender by the advertiser; women were deemed more valuable and

therefore more expensive to reach with the ad so the advertiser sent more ads to men to balance

the average cost per person (Lambrecht and Tucker 2019). The economic valuation of

12
individuals happened to be linked to the underrepresentation of women in STEM careers, which

lead to the “accidental” discrimination (Lambrecht and Tucker 2019).

There are clearly many statistical biases unique to AI models, all of which can lead to

discriminatory or biased decision outcomes. These statistical biases are not addressed in the

existing anti-discrimination laws as most were created before the AI models of today were

developed. Furthermore, it is easy for organizations to justify discrimination in AI models, and

difficult to prove discriminatory intent in a court of law under the existing laws. Ultimately, this

cultural lag means organizations continue to proliferate bias and discrimination through AI even

if they adhere to the law.

The Legal Paradox: Why Anti-Discrimination Laws Could Dissuade Organizations from

Attempting to Reduce AI Discrimination

In addition to being ineffective at preventing discriminatory AI decision outcomes, I propose that

anti-discrimination laws could do additional harm by falsely leading organizations into believing

they are behaving ethically, which could prevent them from taking measures to reduce

discrimination. I discuss this below in the context of procedural and distributive justice.

Most decision outcomes predicted by an AI model can be simplified to a simple binary decision,

for which a positive decision would lead the organization to perform some action based on the

decision outcome that would benefit the individual, and a negative decision would be detrimental

to them. Regardless of the positivity or negativity of the outcome, individuals can assign

different levels of fairness to the decision outcome (Walker et al. 1979). Fairness, also referred to

as justice, can be further broken down into procedural justice, and distributive justice (Walker et

al. 1979). Procedural justice refers to the belief that the process used to arrive at a decision

13
outcome is fair and satisfying in itself (Walker et al. 1979). Distributive justice refers to the

fairness and equity of the decision outcome, in relation to what is thought to be deserved (Walker

et al. 1979). Both these types of justice are in practice measured by the perceptions of

individuals, either those impacted by the decision outcome, those involved in the decision-

making process, or those not directed involved in the decision-making process or directly

affected by its outcome (Gilliland 1993).

Linking the measures of fairness to the AI-based decision-making process, procedural justice is

the fairness perception of the modelling process from the “Purpose” stage through the

“Validation and Approval” stage and the “Monitoring and Review” stage, whilst distributive

justice is the fairness perception of the “Decision Outcome” itself (see Figure 2).

Perceptions of distributive justice should only depend on whether the decision outcome is

positive or negative for the individual impacted, and the outcome fairness based on what the

decision maker believes the individual deserves (Gilliland 1993). But that may not always be the

case; when procedural justice is high, both individuals involved in the decision-making process

and those impacted by the decision outcome have been shown to have higher perceptions

14
distributive justice, regardless of the positive or negative nature of that outcome, and what was

believed to be deserved (Folger and Martin 1986).

Applying this theory to the AI-based decision-making process, I propose that existing anti-

discrimination laws could artificially inflate perceptions of distributive justice, even though, as

discussed, they do not always prevent discrimination in the decision outcome. Many

organizations who have been found to have proliferated discrimination through their AI

including Amazon (Dastin 2018), Microsoft, and IBM (Cossins 2018) have done so whilst

adhering to the law, exemplifying how organizations may not be aware of the law’s

ineffectiveness at preventing discrimination. I propose discrimination occurs because

organizations perceive the decision outcome fairness to be higher than it is, as the anti-

discrimination laws increase their perceptions of procedural justice of the AI development

process. This effect could dissuade individuals within organizations from investigating the

discriminatory impact of decision outcomes or dissuade them from further attempting to reduce

discrimination due to their misinformed perceptions of fairness. A similar inflationary effect has

been shown by the impact of codes of conduct on the ethical perceptions of organizations;

simply having a code of conduct, regardless of the ability of employees to recall its content,

increases the ethical perceptions of the organization (Adams et al. 2001).

Preventing Discriminatory AI

Apart from regulation by law, which as discussed currently has several shortcomings in its

ability to prevent discrimination in AI, there have been a variety of additional methods proposed

that could help organizations reduce discrimination. There is unfortunately no consensus on

which methods are most effective at reducing discrimination in practice (Erwin 2011; House of

Commons Science and Technology Committee 2018; World Economic Forum Global Future

15
Council on Human Rights 2016-18 2018), and as such I will present the leading methods and

discuss some of their potential benefits and risks. The methods I discuss attempt to prevent

discriminatory AI by directly addressing the statistical biases, identifying the discriminatory

decision outcome before it can impact individuals, or promoting ethical behaviour. I present the

prevention methods in relation to the AI-based decision-making process, and the

recommendations are purposely not highly technical so they can be easily implemented by

managers.

Screening the Decision Outcome

Experts agree that if an organization can only afford to do one thing to prevent AI

discrimination, it should be to screen every decision outcome from an AI-based decision-making

process (Barocas et al. 2018; Cowgill and Tucker 2017; Custers et al. 2013). This is a reactive

tactic that attempts to “catch” discriminatory AI outcomes in the “Validation and Approval”

stage before individuals are impacted (see Figure 3 for a reminder of the AI-based decision-

making process). Measuring discrimination is itself a challenge; organizations will need to define

discrimination and fairness in relation to their existing organizational values and the relevant

laws first, and then work with technical AI experts to translate those definitions that the AI

model can understand. The Association of Computer Machinery (ACM) does not condone

discrimination based on age, colour, disability, ethnicity, family status, gender identity, military

status, national origin, race, religion or belief, sex, political affiliation, sexual orientation in their

ACM Code of Ethics and Professional Conduct: Draft 2 (ACM Ethics 2018). This list can be

used in addition to the existing laws to help organizations define discrimination. Clearly,

screening each AI outcome is a reactive tactic, but there are additional measures that

organizations can put in place to address the statistical biases and other drivers of discrimination.

16
Minimizing Statistical Biases

A better solution would be to include additional anti-discrimination measures that help to reduce

each statistical bias discussed across the AI-based decision-making process.

Purpose

In this initiation stage, organizations discuss what the purpose of the project is, and how AI can

be used to help predict a relevant decision outcome. In this preliminary stage the organization

should evaluate the potential impact of the proposed project, for which IT ethics experts

recommend the use of a values-by-design approach (Crawford and Calo 2016; Spiekermann

2016). This is a framework that helps organizations proactively identify and address the ethical

implications of a technology, in advance of the technical development, by gathering input from

stakeholders (Spiekermann 2016). Organizations already using a values-by-design approach, or a

similar values-based design process for other technologies can adapt these for artificial

intelligence. Values-based designs does not usually question the ethical nature of the

technology’s existence though, and instead focus on whether the attributes of the proposed

technology align to the organization’s ethical values, so organizations should also consider

17
during the purpose stage whether or not the AI model should be used at all based on its potential

to discriminate (Crawford and Calo 2016).

Data Collection

When collecting data, organizations should ensure that their sample accurately represents the

population it will impact, and resample or oversample protected classes as needed to ensure

equality (Barocas and Selbst 2016; Custers et al. 2013; Williams et al. 2018). For example,

instead of only using the data of past borrowers to predict credit default, a bank should use data

that is representative of the entire population of applicants (Banasik et al. 2003). Organizations

must also attempt to gather as many protected attributes as legally allowed so that they can test

the fairness of the decision outcomes across protected classes (Andreeva et al. 2004; Chan and

Seow 2013). When attempting to correct past prejudices in the data, observations can be

relabelled or recoded to reflect the current ethical norms of the organization (Barocas and Selbst

2016). Relabelling past data will allow organizations to use existing data sources, however it

may not always be clear what the decision outcome would have been if historical prejudice was

not present, sometimes making it difficult to determine the appropriate relabelling scheme

(Barocas and Selbst 2016). During the data collection stage, organizations should also determine

what level of data granularity is required, and what the decision outcome variable and cut-offs

should be to ensure a fair decision outcome, and can do so by testing the fairness achieved

through different combinations of granularity, decision outcomes variables, and cut-offs

(Barocas and Selbst 2016).

Model Development

18
During this stage, the algorithm, or algorithms used to predict the decision outcome are selected

by technical project members. Algorithm choice has ethical implications because they have been

shown to produce a range of discriminatory outputs depending on the business problem; an

algorithm that is fine in one setting may be extremely discriminatory in another (Custers et al.

2013). Furthermore, algorithms vary in their opacity and complexity, leading to different levels

of explainability, which means different abilities to prevent discriminatory decision outcomes

(Burrell 2016). In order to prevent discrimination, organizations should not use models that are

too complex for technically trained human experts to understand, those that might traditionally

be referred to as “black box” algorithms, as the complexity will not allow them to determine the

drivers of discrimination (Burrell 2016). This applies to both internally and externally developed

algorithms, as organizations will be held responsible and accountable for the outcome no matter

who built the AI model (Martin 2018).

Validation and Approval

This stage is in place to ensure the AI model works as intended, and that the decision outcome

from the algorithm is fair. It should include a test-and-learn or sandbox approach, which will

allows the organization to run the algorithm in a simulated environment with real data, but

without impacting any individuals (Arnold and Scheutz 2018; O’Neil 2016). It should also

include tests of predictability and replicability to ensure the model will perform as intended in a

dynamic environment (Burrell 2016). It is at this point organizations can screen the decision

outcome, and measure its impact to ensure the model is not discriminating against any protected

classes; this tactic as mentioned, is favoured among researchers as way to ensure discrimination

is prevented in AI models (Barocas et al. 2018; Cowgill and Tucker 2017; Custers et al. 2013).

Lastly, during this stage all AI models should be approved by a human as an additional safe-

19
guard against bias (Davenport and Katyal 2018; Vamplew et al. 2018; World Economic Forum

Global Future Council on Human Rights 2016-18 2018).

Monitoring and Review

The AI model and the decision outcome should continue to be monitored and reviewed after

“Validation and Approval” to ensure it is performing as the organization intended, across

changes in data, time, and interactions with external factors. If necessary, this stage acts as a

feedback loop so changes can be made to the “Purpose”, “Data Collection”, “Model

Development”, or “Validation and Approval” stages in order to further reduce discrimination in

the decision outcome (Barocas et al. 2018). Monitoring and reviewing the decision outcome is

currently the only method to diagnose inadvertent biases due to external factors, such as the

STEM career ad example discussed, as the discriminatory impact is likely not visible by testing

the decision outcome in a sandbox or other test-and-learn environment (Lambrecht and Tucker

2019). Monitoring and review will also help to discover any new biases as AI technology

evolves.

Unfortunately it would be naïve to state that all discriminatory decision outcomes could be

retraced to a clear statistical bias, or that all biases could be removed completely (Cowgill and

Tucker 2017), due to the opaqueness and complexity of AI algorithms (Burrell 2016), and the

technology’s continued evolution (Campolo et al. 2017), but measuring the fairness of the

decision outcome and putting in place the recommended methods to reduce statistical biases will

help fill the existing gaps in the anti-discrimination laws, a first step in preventing all AI bias and

discrimination (Cordeiro 1997).

Additional Methods to Reduce Discrimination

20
To further prevent AI bias and discrimination, organizations can put in place methods not

specific to AI. Organizations can increase organizational diversity, particularly by increasing the

number of individuals from protected groups and disciplines outside AI (Campolo et al. 2017;

World Economic Forum Global Future Council on Human Rights 2016-18 2018). Organizations

can also invest in employee education on AI bias; simply making employees more aware of

ethical issues can lead more ethical behaviour (Cordeiro 1997; World Economic Forum Global

Future Council on Human Rights 2016-18 2018).

A code of conduct can also be developed to address how employees are expected to behave in

order to reduce discrimination specific to AI, and can include other ethical AI initiatives such as

accountability and responsibility (Martin 2018), explainability (Burrell 2016), autonomy (Etzioni

and Etzioni 2016), and privacy (Martin 2015). Despite their popularity, there is however mixed

evidence on the efficacy of codes of conduct in changing ethical behaviour (Adams et al. 2001).

Researchers from the Association of Computer Machinery have shown that simply having

software engineers read a code of conduct does not change their behaviour (McNamara et al.

2018), while others have shown that behaviour change depends on the content of the code (Erwin

2011), the frequency and quality of communication about it, as well as the implementation by

both local and senior management (Kaptein 2011). Organizations should therefore take great

care in the development and implementation of an AI ethics code of conduct to ensure they

impact employee behaviour, and do not simply inflate perceptions of justice as some codes have

been shown to do (Adams et al. 2001), and as I propose the existing anti-discrimination laws

already do.

Discussion

An Ongoing Challenge: Laws That Lead to Discriminatory AI

21
Unfortunately, none of these methods can deal with instances where anti-discrimination laws are

leading to less fair outcomes for protected individuals, such as the prohibited use of gender in

credit scoring (Andreeva and Matuszyk 2018). It is an unpalatable situation, and for now,

organizations must of course adhere to the relevant laws; however, it is their ethical

responsibility to push back on the regulation, in tandem with researchers of the phenomenon to

ensure changes are made. The recommendation is not unjustified, as regulators in Singapore

have recently worked closely with industry organizations to develop new anti-discrimination

laws for the financial services industry to more effectively prevent bias and discrimination in AI

models (Monetary Authority of Singapore 2018). There are of course legal and political

challenges associated with policy change (Barocas and Selbst 2016), but the ultimate goal of

reducing discrimination should serve as strong incentive for organizations and regulators alike.

Current Initiatives to Prevent Discriminatory AI

The gap in the existing anti-discrimination laws applied to AI has not got entirely unnoticed,

with many countries, industries, organizations, and researchers working to promote fairness and

other positive ethical behaviours in artificial intelligence.

As early as February 2017, countries started to announce their AI strategies, some of which

included plans and funding for AI ethics (Dutton 2018). In October 2018 at the 40th International

Conference of Data Protection and Privacy Commissioners, 13 privacy commissions signed the

Declaration on Ethics and Data Protection in Artificial Intelligence including the Canadian (and

their Quebec counterpart), the British, and the European Union groups. The declaration is a list

of guiding ethical principles, like a code of conduct, that covers fairness, bias and discrimination,

accountability, transparency, explainability, privacy, and general empowerment of individuals

22
with respect with AI; and is supported by a new permanent international working group

(Commission Nationale de l’Informatique et des Libertes (CNIL) et al. 2018).

More recently, in April the European Commission launched the Ethics Guidelines for

Trustworthy AI, which attempts to ensure AI systems in the EU uphold the seven key

requirements for Trustworthy AI: human agency and oversight; technical robustness and safety;

privacy and data governance; transparency; diversity, non-discrimination and fairness;

environmental and societal well-being, and accountability (Independent High-Level Expert

Group on Artificial Intelligence Set up by The European Commision 2019). In the United Sates,

the Algorithmic Accountability Act of 2019 was also proposed in April, and would have any

business with gross sales over $50M USD, or with more than 1 million customers, perform

algorithmic impact assessments on their AI models to test for accuracy, fairness, bias,

discrimination, privacy, and security (Wyden and Booker 2019).

Many professional standards and codes of conduct have also been developed including the 2018

ACM Code of Ethics and Professional Conduct: Draft 2, the future of life institute’s Asilomar AI

Principles, and IEEE’s Ethically Aligned Design (ACM Ethics 2018; Dignum 2018; Future of

Life Institute 2017; The IEEE Global Initiative for Ethical Considerations in Artificial

Intelligence and Autonomous Systems. 2016). IEEE has also just announced they are developing

an AI ethics technical certification program to educate computer and data scientists on AI ethics,

which they hope will become part of a larger certification process for AI products, systems, and

services (“IEEE Launches Ethics Certification Program for Autonomous and Intelligent

Systems” 2019).

23
Some organizations have also recognized the gap in the existing anti-discrimination laws;

companies including Google, Microsoft, as well as others outside the tech industry like the Bank

of Nova Scotia, and the Monetary Authority of Singapore have implemented AI ethics codes of

conduct in the last 12 months (Hume et al. 2018; Monetary Authority of Singapore 2018;

Simonite 2018). Consulting firms such as Deloitte, Accenture, and PwC have also developed

their own AI ethics frameworks to promote the responsible use of the technology both internally

and at their clients (Deloitte 2018; Demetriades and McLaughlan 2017; Oxborough et al. 2018).

Other organizations have banded together to form best practice groups, such as the Partnership

on AI, which spans across more than 80 organizations (“The Partnership on AI” n.d.).

Future Research Initiatives

In addition to arguing the ineffectiveness of existing laws, I also propose that the laws could

have an inflationary effect on perceived distributive justice which could dissuade organizations

from attempting to prevent discrimination. This proposition is grounded in theory but remains

speculative and should therefore be tested empirically as a next step. I would suggest an

empirical study should include measurements of perceived justice perceptions of all parties

potentially impacted by the AI-based decision-making process (Walker et al. 1979), including

the individuals impacted by the decision outcome, individuals in the organization that are part of

the AI-based decision-making process, as well as members of the public who are not directly

impacted by the decision outcome but engaged in the general pursuit of fairness and equality.

Additionally, the lack of empirical evidence on the efficacy of mechanisms to prevent

discrimination in AI must change; researchers could follow the example of the Association of

Computer Machinery and test the impact of AI ethics codes, policies and procedures (McNamara

24
et al. 2018). Organizations can help push this research forward by agreeing to take part in studies

of the efficacy of anti-discrimination techniques in changing ethical behaviour.

Conclusion

Organizations that simply adhere to the existing anti-discrimination laws will likely not be able

to prevent the proliferation of discrimination and bias from their AI-based decision-making

processes. I discuss how the existing anti-discrimination laws in North America and Europe are

ineffective when applied to AI as they allow for easy justification of discrimination, make it

difficult to prove intent, and ignore forms of statistical bias which are unique to AI models. In

addition, I propose that the existing laws could lead to artificially inflated perceptions of

distributive justice which could paradoxically dissuade organizations from attempting to reduce

discrimination. I recommend that in order to prevent discrimination, organizations should screen

all decision outcomes from AI-based decision-making processes and put in place specific

mechanisms to reduce statistical bias. In addition to the AI specific anti-discrimination measures,

organizations can work to increase the diversity of teams working with AI and educate

employees on the potential ethical issues related to the technology. These anti-discrimination

initiatives can be combined with other ethical initiatives in an AI ethics code of conduct to

specify how employees should act when faced with the ethical implications of AI to help prevent

future bias and discrimination.

25
References

ACM Ethics. (2018). 2018 ACM Code of Ethics and Professional Conduct: Draft 2.

Adams, J. S., Tashchian, A., & Shore, T. H. (2001). Codes of Ethics as Signals for Ethical

Behavior*. Journal of Business Ethics, 29, 199–211.

Andreeva, G., Ansell, J., & Crook, J. (2004). Impact of anti-discrimination laws on credit

scoring. Journal of Financial Services Marketing, 9(1), 22–33.

doi:10.1057/palgrave.fsm.4770138

Andreeva, G., & Matuszyk, A. (2018). The Law of Equal Opportunities or Unintended

Consequences? The Impact of Unisex Risk Assessment in Consumer Credit. SSRN

Electronic Journal, 1–32. doi:10.2139/ssrn.3212702

Arnold, T., & Scheutz, M. (2018). The “big red button” is too late: an alternative model for the

ethical evaluation of AI systems. Ethics and Information Technology, 20(1), 59–69.

doi:10.1007/s10676-018-9447-7

Banasik, J., Crook, J., & Thomas, L. (2003). Sample selection bias in credit scoring models.

Journal of the Operational Research Society, 54(8), 822–832.

doi:10.1057/palgrave.jors.2601578

Barocas, S., Hardt, M., & Narayanan, A. (2018). Fairness and Machine Learning: Limitations

and Opportunities. https://fairmlbook.org/. Accessed 10 December 2018

Barocas, S., & Selbst, A. D. (2016). Big Data’s Disparate Impact. 104 California Law Review

671.

Bart, V. L. (2017). Machine learning: A revolution in risk management and compliance? Journal

26
 of Financial Transformation, 45, 60–67. https://ideas.repec.org/a/ris/jofitr/1592.html

Blöchlinger, A., & Leippold, M. (2006). Economic benefit of powerful credit scoring. Journal of

Banking and Finance, 30(3), 851–873. doi:10.1016/j.jbankfin.2005.07.014

Bower, A., Niss, L., Sun, Y., & Vargo, A. (2018). Debiasing representations by removing

unwanted variation due to protected attributes. In Proceedings of the 35th International

Conference on Machine Learning. Stockholm, Sweden.

https://www.fatml.org/media/documents/debiasing_representations.pdf

Brynjolfsson, E., Mitchell, T., & Rock, D. (2018). What Can Machines Learn, and What Does It

Mean for Occupations and the Economy? AEA Papers and Proceedings, 108, 43–47.

doi:10.1257/pandp.20181019

Burrell, J. (2016). How the machine “thinks”: Understanding opacity in machine learning

algorithms. Big Data & Society, January-Ju, 1–12. doi:10.1177/2053951715622512

Campolo, A., Sanfilippo, M., Whittaker, M., & Crawford, K. (2017). AI Now 2017 Report, 36.

https://ainowinstitute.org/AI_Now_2017_Report.pdf

Canadian Human Rights Act (2017). http://canlii.ca/t/52zkk. Accessed 21 August 2018

Čerka, P., Grigiene, J., & Sirbikyte, G. (2015). Liability for damages caused by artificial

intelligence. Computer Law and Security Review, 31(3), 376–389.

doi:10.1016/j.clsr.2015.03.008

Chan, W. L., & Seow, H. V. (2013). Legally scored. Journal of Financial Regulation and

Compliance, 21(1), 39–50. doi:10.1108/13581981311297812

Commission Nationale de l’Informatique et des Libertes (CNIL), European Data Protection

27
 Supervisor (EDPS), & Garante per la protezione dei dati personali. (2018). Declaration on

Ethics and Data Protection in Artificial Intelligence. Brussels. https://icdppc.org/wp-

content/uploads/2018/10/20180922_ICDPPC-40th_AI-Declaration_ADOPTED.pdf.

Accessed 12 December 2018

Cordeiro, W. P. (1997). Suggested Management Responses to Ethical Issues Raised by

Technological Change. Journal of Business Ethics, 16(12/13), 1393–1400.

Cossins, D. (2018). Discriminating algorithms: 5 times AI showed prejuidice. New Scientist.

doi:10.1111/j.1740-9713.2016.00960.x

Cowgill, B., & Tucker, C. (2017). Algorithmic Bias : A Counterfactual Perspective. Arlington,

VA. http://trustworthy-algorithms.org/whitepapers/Bo Cowgill.pdf

Crawford, K., & Calo, R. (2016). There is a blind spot in AI research. Nature, 538(7625), 311–

313. doi:10.1038/538311a

Custers, B., Calders, T., Schermer, B., & Zarksy, T. (2013). Discrimination and Privacy in the

Information Society: Data Mining and Profiling in Large Databases.

Dastin, J. (2018). Amazon scraps secret AI recruiting tool that showed bias against women |

Reuters. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-

insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-

idUSKCN1MK08G. Accessed 13 December 2018

Davenport, T. H., & Katyal, V. (2018). Every Leader’s Guide to the Ethics of AI. MIT Sloan

Management Review. https://sloanreview.mit.edu/article/every-leaders-guide-to-the-ethics-

of-ai/. Accessed 12 December 2018

28
Day, M. (2016). How LinkedIn’s search engine may reflect a gender bias. The Seattle Times.

https://www.seattletimes.com/business/microsoft/how-linkedins-search-engine-may-reflect-

a-bias/. Accessed 13 December 2018

Deloitte. (2018). Ethics in the age of technological disruption.

https://www2.deloitte.com/content/dam/Deloitte/ca/Documents/deloitte-analytics/ca-en-

analytics-FCC-true-north-aoda.PDF. Accessed 12 December 2018

Demetriades, C., & McLaughlan, T. (2017). An Ethical Framework for Responsible AI and

Robotics. https://www.accenture.com/gb-en/company-responsible-ai-robotics. Accessed 12

December 2018

Dignum, V. (2018). Ethics in artificial intelligence: introduction to the special issue. Ethics and

Information Technology, 20(1), 1–3. doi:10.1007/s10676-018-9450-z

Dopp, T., & Westbrook, J. (2019). Facebook (FB) Violated Fair Housing Act With Ads, HUD

Charges. Bloomberg. https://www.bloomberg.com/news/articles/2019-03-28/facebook-

violated-fair-housing-act-with-ad-practice-hud-charges. Accessed 13 April 2019

Dutton, T. (2018). An Overview of National AI Strategies – Politics + AI – Medium. Medium.

https://medium.com/politics-ai/an-overview-of-national-ai-strategies-2a70ec6edfd.

Accessed 12 December 2018

Erwin, P. M. (2011). Corporate Codes of Conduct: The Effects of Code Content and Quality on

Ethical Performance. Journal of Business Ethics, 99, 535–548. doi:10.1007/s10551-010-

0667-y

Etzioni, A., & Etzioni, O. (2016). AI assisted ethics. Ethics and Information Technology, 18(2),

29
 149–156. doi:10.1007/s10676-016-9400-6

European Union Agency for Fundamental Rights and Council of Europe. (2018). Handbook on

European non-discrimination law. doi:10.2811/58933

Executive Office of the President National Science and Technology Council Committee on

Technology. (2016). Preparing for the future of Artificial Intelligence. doi:10.1007/s00146-

016-0685-0

Floridi, L., Cave, J., Davis, J., Mittelstadt, B., Raab, C., & Wachter, S. (2017). Written evidence

- The Alan Turing Institute.

http://data.parliament.uk/writtenevidence/committeeevidence.svc/evidencedocument/scienc

e- and-technology-committee/algorithms-in-decisionmaking/written/69165.html). Accessed

12 December 2018

Folger, R., & Martin, C. (1986). Relative deprivation and referent cognitions: Distributive and

procedural justice effects. Journal of Experimental Social Psychology, 22, 531–546.

doi:10.1016/0022-1031(86)90049-1

Future of Life Institute. (2017). Asilomar AI Principles. Future of Life Institue.

https://futureoflife.org/ai-principles/?submitted=1#confirmation

Gilliland, S. W. (1993). The Perceived Fairness of Selection Systems: An Organizational Justice

Perspective. Academy of Management Review, 18(4), 694–734.

House of Commons Science and Technology Committee. (2018). Algorithms in decision-

making. https://publications.parliament.uk/pa/cm201719/cmselect/cmsctech/351/351.pdf

Hume, K., Moore, D., & Zerbs, M. (2018). The AI revolution needs a rulebook. Here’s a

30
 beginning. The Globe and Mail.

https://www.theglobeandmail.com/business/commentary/article-the-ai-revolution-needs-a-

rulebook-heres-a-beginning/. Accessed 12 December 2018

IEEE Launches Ethics Certification Program for Autonomous and Intelligent Systems. (2019).

https://standards.ieee.org/news/2018/ieee-launches-ecpais.html. Accessed 11 December

2018

Independent High-Level Expert Group on Artificial Intelligence Set up by The European

Commision. Ethics Guidelines for Trustworthy AI (2019).

Kaptein, M. (2011). Toward Effective Codes: Testing the Relationship with Unethical Behavior.

Journal of Business Ethics, 99(2), 233–251.

Kastellec, J. P. (2010). The Statistical Analysis of Judicial Decisions and Legal Rules with

Classification Trees. Journal of Empirical Legal Studies, 7(2), 202–230.

doi:10.1111/j.1740-1461.2010.01176.x

Khalil, O. E. M. (1993). Artificial Decision-Making and Artificial Ethics: A Management

Concern. Journal of Business Ethics, 12(4), 313–321.

Kirkpatrick, K. (2016). Battling algorithmic bias. Communications of the ACM, 59(10), 16–17.

doi:10.1145/2983270

Koene, A. (2017). Algorithmic Bias: Addressing Growing Concerns. IEEE Technology and

Society Magazine, (June), 31–32.

Lambrecht, A., & Tucker, C. (2019). Algorithmic Bias? An Empirical Study of Apparent

Gender- Based Discrimination in the Display of STEM Career Ads. Management Science.

31
 doi:https://doi.org/10.1287/mnsc.2018.3093

Marie, M.-B. (2016). History of Antidiscrimination Law: The Constitution and the Search for

Paradigms of Equality. In Discrimination at Work: Comparing European, French, and

American Law (pp. 9–28). Oakland.: University of California Press.

Marshall, K. P. (1999). Has Technology Introduced New Ethical Problems? Journal of Business

Ethics, 19(1), 81–90.

Martin, K. (2015). Ethical Issues in the Big Data Industry. MIS Quarterly, 14(2), 67–85.

Martin, K. (2018). Ethical Implications and Accountability of Algorithms. Journal of Business

Ethics, 0(0), 1–39. doi:10.1007/s10551-018-3921-3

McNamara, A., Smith, J., & Murphy-Hill, E. (2018). Does ACM’s code of ethics change ethical

decision making in software development? In Proceedings of the 2018 26th ACM Joint

Meeting on European Software Engineering Conference and Symposium on the

Foundations of Software Engineering - ESEC/FSE 2018 (pp. 729–733).

doi:10.1145/3236024.3264833

Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of

algorithms: Mapping the debate. Big Data & Society, (July-December), 1–21.

doi:10.1177/2053951716679679

Monetary Authority of Singapore. (2018). Guide to Promote Fairness, Ethics, Accountability

and Transparency (FEAT) in the Use of Artificial Intelligence and Data Analytics in the

Singapore Financial Sector.

Murgia, M. (2019, February). How to stop computers being biased. The Financial Times.

32
 https://www.ft.com/content/12dcd0f4-2ec8-11e9-8744-e7016697f225

O’Neil, C. (2016). Weapons of Math Destruction. (Broadway Books, Ed.). New York: Crown.

Oxborough, C., Egglesfield, L., Cameron, E., & McCargow, R. (2018). The responsible AI

framework. https://www.pwc.co.uk/services/audit-assurance/risk-

assurance/services/technology-risk/technology-risk-insights/accelerating-innovation-

through-responsible-ai/responsible-ai-framework.html. Accessed 12 December 2018

Simonite, T. (2018). Tech Firms Move to Put Ethical Guard Rails Around AI. Wired.

https://www.wired.com/story/tech-firms-move-to-put-ethical-guard-rails-around-ai/.

Accessed 12 December 2018

Spiekermann, S. (2016). Ethical IT Innovation: A Value-Based System Design Approach. (J.

Cantella, Ed.). Boca Raton: Taylor & Francis Group, LLC.

Taylor, W. (2011). Proving Racial Discrimination and Monitoring Fair Lending Compliance:

The Missing Data Problem in Nonmortgage Credit. Review of Banking & Financial Law,

31, 199–264.

The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous

Systems. (2016). Ethically Aligned Design: A Vision for Prioritizing Wellbeing With

Artificial Intelligence and Autonomous Systems, Version 1.

doi:10.1109/IHTC.2017.8058187

The Partnership on AI. (n.d.). https://www.partnershiponai.org/. Accessed 11 December 2018

Vamplew, P., Dazeley, R., Foale, C., Firmin, S., & Mummery, J. (2018). Human-aligned

artificial intelligence is a multiobjective problem. Ethics and Information Technology,

33
 20(1). doi:10.1007/s10676-017-9440-6

Walker, L., Lind, E. A., & Thibaut, J. (1979). The Relation between Procedural and Distributive

Justice. Virgina Law Review, 65(8), 1401–1420.

Wellman, M. P., & Rajan, U. (2017). Ethical Issues for Autonomous Trading Agents. Minds and

Machines, 27(4), 609–624. doi:10.1007/s11023-017-9419-4

Williams, B. A., Brooks, C. F. B., & Shmargad, Y. (2018). How Algorithms Discriminate Based

on Data they Lack: Challenges, Solutions, and Policy Implications. Journal of Information

Policy, 8, 78–115. doi:10.5325/jinfopoli.8.2018.0078

Wilson, H. J., & Daugherty, P. R. (2018). How Humans and AI Are Working Together in 1,500

Companies. Harvard Business Review. https://hbr.org/2018/07/collaborative-intelligence-

humans-and-ai-are-joining-forces. Accessed 14 December 2018

World Economic Forum Global Future Council on Human Rights 2016-18. (2018). How to

Prevent Discriminatory Outcomes in Machine Learning.

Wyden, R., & Booker, C. Algorithmic Accountability Act of 2019 (2019).

Zarksy, T. Z. (2014). Understanding Discrimination in the Scored Society. The Washington Law

Review, 89, 1375–1412.

Zarsky, T. (2016). The Trouble with Algorithmic Decisions: An Analytic Road Map to Examine

Efficiency and Fairness in Automated and Opaque Decision Making. Science Technology

and Human Values, 41(1), 118–132. doi:10.1177/0162243915605575

Zhang, Y. (2018). Assessing Fair Lending Risks Using Race / Ethnicity Proxies Assessing Fair

Lending Risks Using Race / Ethnicity Proxies. Management Science, 64(January 2019),

34
 178–197.

35

View publication stats