Association for Information Systems

AIS Electronic Library (AISeL)

ICIS 2021 Proceedings AI in Business and Society

Dec 12th, 12:00 AM

AI Divide versus Inclusion: An Empirical Evidence from an On-

demand Food Delivery Platform
Yeonseo Kim
College of Business, KAIST, dus1346@kaist.ac.kr

Jiyong Eom
KAIST College of Business, eomjiyong@kaist.ac.kr

Tae Jung Yoon

KAIST, taejung.yoon@kaist.ac.kr

Albert Jin Chung

Mesh Korea, jinwoo.chung@meshkorea.net

Myunghwan Kim
Mesh Korea, myunghwan.kim@meshkorea.net

Follow this and additional works at: https://aisel.aisnet.org/icis2021

Recommended Citation
Kim, Yeonseo; Eom, Jiyong; Yoon, Tae Jung; Chung, Albert Jin; and Kim, Myunghwan, "AI Divide versus
Inclusion: An Empirical Evidence from an On-demand Food Delivery Platform" (2021). ICIS 2021
Proceedings. 5.
https://aisel.aisnet.org/icis2021/ai_business/ai_business/5

This material is brought to you by the International Conference on Information Systems (ICIS) at AIS Electronic
Library (AISeL). It has been accepted for inclusion in ICIS 2021 Proceedings by an authorized administrator of AIS
Electronic Library (AISeL). For more information, please contact elibrary@aisnet.org.
 AI Divide versus Inclusion

AI Divide versus Inclusion:

Evidence from an On-demand Food Delivery
Platform
Short Paper

Yeonseo Kim Jiyong Eom

KAIST KAIST
85, Hoegi-ro, Dongdaemun-gu, Seoul, 85, Hoegi-ro, Dongdaemun-gu, Seoul,
Republic of Korea Republic of Korea
dus1346@kaist.ac.kr eomjiyong@kaist.ac.kr

Taejung Yoon Albert Jin Chung

KAIST Mesh Korea
85, Hoegi-ro, Dongdaemun-gu, Seoul, 418, Teheran-ro, Gangnam-gu, Seoul,
Republic of Korea Republic of Korea
taejung.yoon@kaist.ac.kr jinwoo.chung@meshkorea.net

Myunghwan Kim
Mesh Korea
418, Teheran-ro, Gangnam-gu, Seoul, Republic of Korea
myunghwan.kim@meshkorea.net
Abstract
Despite the growing adoption of AI in business operations, the potential differential
effects of AI adoption on individual workers remain ambiguous. Drawing on the unique
microdata of an on-demand food delivery platform, we investigate the impact of AI
adoption on the labor productivity of delivery workers at different skill levels. Our results
show that AI adoption enhances the workers’ labor productivity by 3% on average. The
enhancement effect was concentrated on low- and middle-skilled drivers (6.5% and 5%,
respectively) without a significant effect on the high-skilled. Furthermore, the labor
productivity gap involved improvements in overall system efficiency and customer
service quality. Our study provides a piece of empirical evidence that introducing AI to
markets with a significant labor productivity gap may bridge this gap and yield a Pareto
improvement for all stakeholders.

Keywords: Artificial intelligence, on-demand economy, labor productivity, labor inequality,

system efficiency, customer satisfaction, food delivery service

Introduction
Artificial intelligence (AI) is rapidly increasing pervasiveness across almost all business sectors, with an
undeniable impact on labor. The on-demand economy, such as ride-hailing and food delivery services, is a
representative sector leading this transformation with various AI-based technologies to manage the
workforce efficiently and improve service quality for customers. Studies on AI impacts have primarily

Forty-Second International Conference on Information Systems, Austin 2021

1
 AI Divide versus Inclusion

focused on the potential effects of AI on labor, inequality, and productivity at the industry or country level
and its differential impact across jobs (Frank et al. 2019; Seamans and Raj 2018). However, although many
companies introduce their own AI algorithm into their core business tasks, there is scant empirical
literature, particularly at a micro-level, on how AI affects individual workers’ labor.
In this paper, we address this under-explored area by answering three key questions (1) how does AI affect
individual workers’ productivity, (2) how does AI adoption affect the labor productivity gap between
workers at different skill levels, and (3) How do these AI-enabled changes eventually affect overall system
efficiency and service quality?
Using novel data from an on-demand food delivery platform, we find various effects of AI-based automated
dispatch systems on delivery drivers. Our results demonstrate that AI improves the productivity of delivery
drivers by 3% in terms of the number of orders completed per labor hour. Interestingly, this improvement
is concentrated among the low- and middle-skilled workers: the productivity of the low-skilled workers
improves by 6.5% and the middle-skilled workers by 5%, with no significant impact on the high-skilled. The
productivity gain for beginner-to-intermediate workers is caused by increases in so-called stacked orders,
the workers’ practice of stacking orders to minimize idling. These findings suggest that AI has the potential
to alleviate inequality in labor productivity between workers. Our study also reveals that AI adoption
increases the number of orders completed per labor hour on the platform and decreases the total waiting
time of the customers. These findings collectively suggest that AI adoption can enhance both system-level
efficiency and customer service quality.
Our research makes three key contributions to the literature. First, our work provides empirical evidence
of the impact of AI on labor productivity at the individual worker level. In contrast to previous industry-
and country-level studies on the impact of AI on labor, we draw on rich individual workforce-level
microdata from an on-demand food delivery platform that has adopted AI for its business operations,
investigating AI’s impact on productivity and associated market changes. Second, our study is among the
first to uncover AI’s implications for inequality in the labor market: AI has the potential to lessen inequality
in productivity between workers at varying skill levels. The implication is that AI may bridge the existing
productivity gap between workers by helping low-skilled workers shorten the learning curve without
statistically significant impacts on high-skilled workers. Third, we highlight that the productivity gains from
AI adoption can further benefit the system and customers as a whole. Introducing AI to an on-demand
platform with a significant labor productivity gap can be a Pareto improving strategy.

Related Literature
As AI is gaining increasing attention from the business community, an in-depth examination of the impact
of AI is actively underway. The mainstream research concerns the impact of AI on the nature of labor, such
as employment, productivity, and inequality. However, previous studies relied on the industry- and
country-level measures so that their scope of AI remains broad and the impact on labor is mixed. One
possible reason is the lack of high-quality and high-resolution microdata, essential for estimating the
relationship between the adoption of AI by individual workers and their performance and skills. A more
disaggregated approach is needed to investigate the AI-induced change in the labor market, which occurs
at the individual worker level.
Although the potential impact of AI on employment and inequality issues remains ambiguous, the literature
finds that AI’s impact on workers’ productivity is primarily positive. Previous studies of the effect of robots
(which are physically similar to AI) on the country- and firm-level productivity find significant productivity
growth (Graetz and Michaels 2018; Zanker et al. 2016). A few recent studies on the effect of AI on individual
workers also find that AI increases labor productivity and job skills (Kanazawa et al. 2021; Luo et al. 2021).
The particular AI we examine does not directly compete with workers but replaces one of the skills that
constitute their profession. As AI becomes increasingly sophisticated, it can replace non-routine complex
cognitive tasks and even outperform humans (Chui 2017). Because the skills that AI replaces or
supplements are too complex for human cognitive abilities, AI with strengths in processing big data and
learning complex latent patterns hidden in data is likely to improve human decisions. We hypothesize as
follows:
H1 The adoption of AI has a positive effect on labor productivity.

Forty-Second International Conference on Information Systems, Austin 2021

2
 AI Divide versus Inclusion

Previous literature on the impact of AI on labor considers differential impacts on jobs. It takes the task-
based approach that each specific technology substitutes or complements specific tasks with varying
bundles of task requirements of different job titles differentiating the technology’s impact. However, a few
recent studies with access to worker-level data demonstrate the varying impacts of new technology within
jobs. They show substantial variation exists between workers within the same occupation depending upon
their socio-economic characteristics, job-related information, and competencies (Kanazawa et al. 2021; Luo
et al. 2021).
Thus, we expect AI’s heterogeneous impact on labor productivity depending upon workers’ current task
proficiency level. Although low-skilled workers may understand the basic concept of tasks, they lack tacit
knowledge on the nature of tasks and associated workplace conditions. However, the structural nature of
the on-demand economy or namely, the gig economy, makes it difficult for these unskilled workers to
improve workplace skills rapidly. Because anyone can start a job without any contractual difficulties in this
gig economy, high-skilled workers are fewer in supply than low-skilled workers and thus maintain greater
bargaining power than the latter in the labor market (Amit 2021). Moreover, there is no direct intervention
from managers or platforms and no training programs for workers. Consequently, for unskilled beginners
in such a harsh situation, AI that substitutes or complements a specific task might improve their skills.
By contrast, high-skilled workers with significant experience and know-how on task skills can hold a
dominant position among workers. Given this, a marginal improvement in their skills from adopting
human-like or human-outperforming AI might be small or insignificant. Furthermore, existing literature
finds that competent workers tend to have a greater aversion to AI algorithms (Dietvorst et al. 2015) and
more desire for autonomy and control, which induces more aversion to machines (Burton et al. 2020). The
potential aversion to the AI or self-trust of high-skilled workers might obstruct further learning and
performance gains from AI adoption. Moreover, the recent controversy on “algorithmic control” of AI—the
process of maximizing system efficiency incurring unexpected or undesirable outcomes—can affect
experienced workers. As AI algorithms are not necessarily designed to benefit a particular group of workers,
it is likely to redistribute the excessive benefits of high-skilled workers or even penalize them for achieving
system-wide efficiency and fairness (Bokányi and Hannák 2020; Wood et al. 2019). This line of reasoning
argues in favor of the marginal impact of AI on workers with expert skills. Thus, we hypothesize on the
impact of AI on productivity by skill levels as follows:
H2 The impact of AI adoption on productivity improvement is higher for low-skilled workers than high-
skilled workers

Empirical Setting: On-Demand Food Delivery Platform

Platform Overview
We collaborate with an on-demand last-mile delivery company in Korea. Unlike food delivery platforms
that provide channels for end-users to place orders (e.g., Uber Eats and Doordash), the company provides
delivery services to the subscribed restaurants.
The default mode of order assignment for delivery workers is manual dispatch. When a delivery worker
signs into the driver app, the app displays a list of orders available to the driver at the geographical location
and information about origin-destination points, delivery distance, and earnings. After choosing an order
from the list, the worker goes pick up the packed food from the store and delivers it to the customer. The
app displays orders simultaneously to the other drivers in the same area, and the order becomes assigned
on a first-come-first-serve basis.

Shifts and Stacked Orders

Delivery workers can take the next orders after completing the current order or during undertaking it,
whichever is beneficial to them. The latter case of combining orders is called the stacked orders. As the
practice allows delivery workers to commence the next delivery before completing the current one, it
reduces idling and, in most cases, benefits them. Our study examines the patterns of order stacking and
their impacts by setting the unit of analysis at a “shift,” a series of order(s) that workers complete in one
continuous traversal.

Forty-Second International Conference on Information Systems, Austin 2021

3
 AI Divide versus Inclusion

For ease of exposition, we introduce a shorthand notation for order processing within a shift. The delivery
process of a single order, “Order 1,” is expressed as “assign1-pickup1-delivery1” (Order 1 in a shift). Not
provided with the next order before completing Order 1, the worker starts idling searching for the following
order. Alternatively, the worker may carry out more than one order within a shift by choosing to stack them
one another. For example, the driver may perform the task in the following sequence “assign1-assign2-
pickup1-pickup2-delivery1-delivery2” (Orders 1 and 2 in a shift). A shift is completed when the worker
processes all assigned orders in the queue and starts searching for new orders.
Order stacking can be delivery workers’ key strategy to maximize labor productivity and earnings. By
choosing orders that share origins or destinations in nearby areas, they can save total time for completing
multiple orders and increase productivity. Moreover, stacking orders prevents the workers from spending
too much time searching for the following orders while idling between shifts.

AI-Based Automated Dispatching System

Delivery operations under the manual dispatch system have some limitations. It is a non-trivial task for the
drivers to find an optimal sequence of orders. They should consider various factors such as the driver's
current location, pick-up and drop-off location, merits of executing each order as determined by per
distance delivery fee and expected delivery time, expected revision to the ongoing route by accepting
additional orders, and other factors. Furthermore, an inherent competition among the drivers in order
dispatching adds to the complexity. During the peak (i.e., lunch or dinner), as demand surges and the supply
of drivers increases, the competition for order assignment accelerates, requiring drivers to make decisions
for the best orders within seconds. The drivers cannot afford a moment to thoroughly consider the factors
above or the profitability of an order. The company introduced an AI-based automated dispatch system to
relieve these limitations of the manual dispatch system.
The company’s AI automated dispatch system uses past data on driver's order preference given each driver's
situation. The automated dispatch system learns the probabilistic distribution of the drivers order
preference, given a list of orders nearby. The AI automated dispatch system is designed to emulate the
driver's preference and automatically assigns individual orders that best match each driver’s interest. The
drivers can freely turn on and off the AI automated dispatch mode on the company’s mobile app. When the
AI automated dispatch mode is turned on, it automatically assigns an order to the driver when the driver is
considered to be the best match for a newly created order. Such AI-dispatched order is expected to be better
performed by the matched driver (than any other competing driver) given their location at the time of
assignment and expected route given the list of orders in-progress. Therefore, the AI is expected to improve
the drivers’ productivity by reducing time spent on searching for orders allowing them to focus wholly on
completing the order delivery. The AI dispatch system maintains fairness between the AI adopters and non-
adopters by enforcing the adopters to complete the assigned order unless there is a specific reason. This
discourages the adopters from selectively gaming the system for further advantage.

Data
In October 2020, the company officially launched the AI dispatch system in Busan, Korea. We measure the
new system’s impact on driver performance using a before-and-after design based on delivery records of
individual drivers from one month before to one month after the launch date (September to November
2020). Each delivery record includes the driver ID, geographical locations (i.e., latitude and longitude) of
the store and the delivery destination, timestamp for each delivery process (i.e., creation, assignment, pick
up, and delivery), delivery fee, and whether the order was assigned automatically by the AI. The data contain
900+ drivers with at least one delivery record during the two months. We retain approximately 600 drivers
who joined the platform before the data period and had delivery records after AI introduction.

Methodology and Results

Identification Strategies
The official release of the AI automated dispatch system in the platform presents a natural shock in our data
period. The main challenge for estimating the effects of AI is the endogeneity problems inherent in the
driver's adoption decision. We address the potential selection bias present among the AI adopters. For

Forty-Second International Conference on Information Systems, Austin 2021

4
 AI Divide versus Inclusion

instance, those who have more difficulties taking preferred orders are more likely to adopt the AI-based
automated dispatch service. As such, drivers who have adopted AI would represent a biased sample,
resulting in an overestimation of the effect of AI adoption. We address the potential endogeneity problem
by performing a difference-in-difference (DID) estimation based on a sample obtained from propensity
score matching (PSM).
We have performed a propensity score matching of the original sample to obtain balanced pairs of treatment
subjects (i.e., AI adopters) and control subjects (i.e., AI non-adopters) within the sampling period.
Observations from the first sampling month, the immediate period before introducing the AI automated
dispatch system, were used to identify the non-adopter group that is statistically similar to the adopter
group. We employ a logistic regression model to represent the binary decision-making of adopting the AI
automated dispatch system. The matching covariates include individual workers’ job performance
characteristics, such as the average number of unique restaurants where the worker started delivery during
the day, the average number of days worked in a week, average time spent for assignment, pickup, and
delivery per order, average per distance service fee of orders, the average number of shifts during the day,
the average number of orders completed in an hour, and average idle duration per day. Our matching
specification for the main results is one-to-one matching without replacement under the caliper size 0.2
times the standard deviation of the propensity scores. We verify our matching performance by comparing
the group means of the covariates and the dependent variable. The t-test results indicate that, after
matching, a valid null hypothesis that the treated and control groups have the same population means for
all variables cannot be rejected at the 1% level. This procedure produced a dataset of 354 drivers.
We then estimate the impact of AI adoption on drivers’ labor productivity and the moderating effect that
their proficiency level has on the impact of AI adoption. The proficiency of each driver is quantified using
the average completed orders per labor hour in the pre-adoption period (pretreatment daily productivity).
We discretize the proficiency level into three: “novice,” “intermediate,” and “advanced.”
Category and Definition Mean SD
variable
Shift-level
Order count Number of delivery orders in a shift 3.183 3.034
Shift duration Time spent from the assignment of the first 33.416 27.802
order to the drop-off of the last order in a shift
(in minutes)
Per distance Average delivery fee per kilometer for orders in a - 0.661
delivery fee shift (in KRW) (𝑥 mean)
Day-level
Daily productivity Average number of orders completed in an hour - 0.184
(𝑥 mean)
Order count Number of orders delivered in a day - 0.378
(𝑥 mean)
Labor hours Total labor hours in a day (in minutes) 378.175 125.702
Idle time ratio Ratio of hours that the riders spend idling 0.189 0.094
without orders among labor hours in a day
Notes: Due to confidentiality agreement with the company, we do not disclose the mean values for
variables, per distance delivery fee, daily productivity, and order count. The standard deviation of
these variables is expressed as 𝑥 times the actual mean values.

Table 1. Variable Definition and Summary Statistics (Individual Driver Level)

Our study analyzes the impact of AI adoption both at the shift and day levels. The list of outcome variables
considered in each level of analysis and their detailed descriptions are presented in Table 1. The estimation

Forty-Second International Conference on Information Systems, Austin 2021

5
 AI Divide versus Inclusion

is based on the following difference-in-difference-in-difference (DDD) model (or DID model excluding the
three-way interaction term):

𝑙𝑜𝑔&𝑌!"# ( = 𝛽𝐴𝑓𝑡𝑒𝑟# × 𝑇𝑟𝑒𝑎𝑡!" + 4 𝛾$ 𝐴𝑓𝑡𝑒𝑟# × 𝑇𝑟𝑒𝑎𝑡!" × 𝑃𝑟𝑜𝑓𝑖𝑐𝑖𝑒𝑛𝑐𝑦!"$ + 𝛼# + 𝜏! + 𝜇" + 𝜖!"# ,

$

𝑘 ∈ {𝑖𝑛𝑡𝑒𝑚𝑒𝑑𝑖𝑎𝑡𝑒, 𝑎𝑑𝑣𝑎𝑛𝑐𝑒𝑑}
where 𝑖 is a driver, 𝑗 is a management branch where the driver 𝑖 belongs, and 𝑡 is a shift (or day). 𝑌!"# is the
outcome of interests in each level of analysis, and 𝐴𝑓𝑡𝑒𝑟# is a binary variable that indicates the period after
the official launch of the AI automated dispatch system. To avoid a possible endogeneity issue related to the
individual adoption period, we set the post-treatment period as the time after the official launch of AI
instead of the time after the adoption of individual drivers (Narang and Shankar 2019). 𝑇𝑟𝑒𝑎𝑡!" is a
treatment dummy variable that equals 1 if a driver has ever completed the order assigned by the AI
automated dispatch system and 0 if otherwise. We also include individual driver-specific fixed effects (𝜏! ),
management branch fixed effects (𝜇" ), and a series of day dummies (𝛼# ) to control for both unobservable
heterogeneities across drivers and systematic longitudinal changes in time trends.
The parameters of interest are the coefficients of the two-way interaction term (𝛽) and the three-way
interaction terms (𝛾$ ), which capture the effect of AI adoption on the dependent variables and the marginal
impact for drivers with intermediate and advanced levels of skill relative to those in the novice group.
The DID approach premises the assumption of parallel pre-treatment trends. To verify whether it holds, we
carried out a “placebo test” using pre-treatment data. In particular, we set a placebo treatment at the mid-
point of our pre-treatment period. We estimate the same DID model for all dependent variables we used in
our analyses. For all variables, we failed to reject the null hypothesis of no treatment effect for placebo
treatments. This indicates that the pre-treatment trends are statistically equivalent across control and
treatment groups.

AI impact on Productivity and Impact by Proficiency Level

Table 2 presents the impact of the AI automated dispatch system on daily productivity measured by the
average completed orders per labor hour for a day. Column (1) reveals that the average gains in productivity
is approximately 3% (p < 0.01). This result implies that the drivers who adopt the AI automated dispatch
system can deliver slightly more orders per hour than those who never adopt it. Thus, our empirical result
supports Hypothesis 1.

Daily productivity
(1) (2)
After × Treat 0.030*** 0.065***
(0.009) (0.013)
After ×Treat × Intermediate -0.025*
(0.013)
After × Treat × Advanced -0.074***
(0.014)
Observations 15,155 15,155
Adj. R squared 0.626 0.627
Notes: Robust standard errors clustered at rider level in parentheses.
***p < 0.01; **p < 0.05; * p < 0.1
Table 2. AI Impact on Productivity and Impact by Proficiency Level

Column (2) of Table 2 presents the productivity changes by skill level and reveals that productivity
improvement is concentrated on the low- and middle-skilled drivers in the novice and intermediate groups.

Forty-Second International Conference on Information Systems, Austin 2021

6
 AI Divide versus Inclusion

The results confirm that AI adoption improves productivity by as much as 6.5% (𝛽 = 0.065, p<0.01) for the
novice drivers and 4% (𝛽 + 𝛾!&#'()'*!+#' = 0.040, p<0.01) for the intermediate drivers, but it does not have
a significant impact on high-skilled drivers (𝛽 + 𝛾+*,+&-'* = −0.009, p>0.1). This implies that the AI
potentially reduces the existing productivity gap across workers and improves inequality in productivity in
this labor market. Our results demonstrate that, whereas the low- and middle-skilled drivers still lack skills
related to order assignment and delivery operation and some of these are improved by adopting the AI
automated dispatch system, the high-skilled drivers have already mastered or are at least fully confident in
those skills such that there is no additional benefit that the current level of AI can provide. Consequently,
our results support Hypothesis 2.
Of remaining concern is the source of these productivity gains. We examined this phenomenon by
conducting analyses on additional variables of shift (Columns (1)—(6) of Table 3) and day levels (Columns
(1)—(6) of Table 4). Because drivers stack nearly half of the orders with other orders, shift-level analysis is
essential for understanding the impact of the AI on drivers’ core skill to maximize their productivity. As
depicted in Columns (2), we find evidence that the AI enables low-skilled drivers to operate a longer shift
with more orders. The order count in a shift increases in the novice group by 6.9% (p < 0.01). The impact
for intermediate drivers is still positive and the impact for advanced drivers turns out negative, but neither
are statistically significant (𝛽 + 𝛾!&#'()'*!+#' = 0.013 𝑎𝑛𝑑 𝛽 + 𝛾+*,+&-'* = −0.03, both p > 0.1). Similarly,
the duration of a shift increases in the novice group by 3.5% (p < 0.05), but the impacts for intermediate
and advanced drivers are statistically insignificant (𝛽 + 𝛾!&#'()'*!+#' = −0.011 𝑎𝑛𝑑 𝛽 + 𝛾+*,+&-'* = −0.023,
both p > 0.1). This result implies that AI helps drivers enhance their core skill to maximize their productivity
(i.e., order stacking) and this role only significantly works for novice drivers.
Column (6) of Table 3 presents the additional benefit of adopting AI: improving per distance delivery fee of
an order, which is also valid for only low-skilled drivers. Without the AI adoption, the low-skilled drivers
tend to choose orders with low per distance delivery fee because, in a highly competitive market, a few high-
skilled workers would preempt orders with high per distance delivery fee. In this scenario, when drivers
delegate the dispatching task to an AI that only follows the objective function of the algorithm, the orders
with high per distance delivery fee biased toward the expert drivers would be redistributed and beginners
would have the greatest benefit.
In day-level analysis, we focus on the changes in daily labor hours and their idle time ratio. Columns (1)—
(4) of Table 4 reveal that the productivity gains we confirmed earlier do not lead to increases in the number
of completed orders in a day or daily labor hours. Instead, the AI reduces the idle time spent searching for
orders without any order at hand, and the impact diminishes and becomes insignificant as the driver skill
level increases (Column (6) of Table 4).
Shift-level Analysis
Order count Shift duration Per distance delivery fee
(1) (2) (3) (4) (5) (6)
After × Treat 0.013 0.069*** -0.002 0.036** 0.006 0.032**
(0.012) (0.017) (0.013) (0.015) (0.011) (0.014)
After × Treat × -0.055*** -0.048*** -0.023
Intermediate (0.018) (0.018) (0.016)
After × Treat × -0.099*** -0.060** -0.048***
Advanced (0.026) (0.024) (0.015)
Observations 171,317 171,317 171,317 171,317 168,256 168,256
Adj. R squared 0.289 0.289 0.353 0.353 0.073 0.073
Notes: Robust standard errors clustered at rider level in parentheses.
***p < 0.01; **p < 0.05; * p < 0.1
Table 3. AI Impact on shift level behaviors

Forty-Second International Conference on Information Systems, Austin 2021

7
 AI Divide versus Inclusion

Day-level Analysis
Order count Labor hours Idle time ratio
(1) (2) (3) (4) (5) (6)
After × Treat 0.012 0.027 -0.018 -0.038 -0.011* -0.025***
(0.033) (0.052) (0.035) (0.056) (0.006) (0.008)
After × Treat × -0.011 0.014 0.013*
Intermediate (0.058) (0.060) (0.007)
After × Treat × -0.032 0.042 0.029***
Advanced (0.058) (0.060) (0.008)
Observations 15,155 15,155 15,155 15,155 15,155 15,155
Adj. R squared 0.623 0.623 0.532 0.532 0.521 0.521
Notes: Robust standard errors clustered at rider level in parentheses.
***p < 0.01; **p < 0.05; * p < 0.1
Table 4. AI Impact on day level behaviors

The Impact of AI on System Efficiency and Service Quality

Previously, we found evidence of productivity gains—the number of completed orders per labor hour
increased by 3% on average (Column (1) in Table 2). When we expand and reinterpret this result to the
platform level, the amount of workload that can be handled in an hour with the same number of workers as
now increases by 3%, indicating an improvement in system-level efficiency. Furthermore, the earlier
evidence that the idle time ratio decreased while the total daily labor hours did not change (Column (3) and
(5) in Table 4) implies that the platform can squeeze the most out of labor from workers for the time they
are at the workplace. This also may result in system efficiency gains.
The results thus far demonstrate that AI adoption has benefits both at the individual driver and the system
level. What remains is whether there will be any benefit to consumers. We verified this by estimating the
impact of AI on consumer’s waiting time measured by the average time spent from order placement to
delivery per order in a shift. The result demonstrates that the average waiting time for consumers decreased
slightly by 1.3% (p < 0.1). Consequently, although the impact is limited and trivial, the AI adoption of drivers
appears to provide slight advantages to consumers.
We further verify the robustness of our results by employing alternative matching methods. Specifically, we
use a tighter caliper size of 0.05 times the standard deviation of propensity scores to assess the stability of
results under more stringent matching conditions and conduct the baseline matching procedure with a
dataset excluding drivers who have only used AI once during the data period. We confirm that the results
from alternative matching methods remain consistent with the results shown earlier.

Discussion
Despite the increasing ubiquity of AI in workplace operations, there is a paucity of individual-level empirical
studies on how AI affects labor productivity. Based on the driver-level data from the on-demand food
delivery industry, we investigate the impact of AI on workers’ productivity, its differential impact by worker
skill level, and how these changes in workers eventually affect system-level efficiency and consumer-side
service quality. The results demonstrate that the AI improves the labor productivity measured by the
number of completed orders per hour, concentrated among beginner-level drivers. Accordingly, AI has the
potential to reduce the existing productivity gap among workers, mitigating inequality in the labor market.
We also find that AI-enabled changes in workers lead to more efficient system operation and higher service
quality.
Our findings provide two major implications for platform operators contemplating or already implementing
AI. First, replacing a task with AI can address the skill gap for new workers, allowing them to better adapt

Forty-Second International Conference on Information Systems, Austin 2021

8
 AI Divide versus Inclusion

to their job and workplace environment. The traditional strategy that companies have used to improve the
skills of beginner workers is to educate them through training programs. Skill enhancement through
education, however, requires considerable time and its effectiveness is not always guaranteed. The use of
AI for tasks that beginners find difficult may be the simplest and most obvious way to increase productivity.
This strategy can lower the entry barrier of the labor market and prevent low-skilled workers from being
left behind by the market competition. However, using an AI-based strategy may cause workers to depend
on AI rather than increase their capability for the job. Second, just because AI helps novice workers become
productive does not mean they can become experts overnight. Although our result of a 6.5% increase in
novice worker productivity is non-trivial, some skills cannot be replaced by the current level of AI
technology because the initial productivity gap between novice and advanced workers is about 60% or more.
Nevertheless, this study is not without limitations. First, because our dataset contains one month before
and after launching the new AI system, our study determines the impact of AI for only about a month after
its adoption. If longer-term data are available, it would provide deeper understanding by enabling us to
analyze how the AI's impacts on productivity by worker skill level vary over time. Second, our empirical
results on AI concern a particular task (delivery job) in a particular industry (on-demand food delivery).
The context of our research is characterized by competitive labor supply and a significant skill gap between
workers. It would be interesting to examine whether AI has similar effects in other settings.

Acknowledgements
The authors thank Mesh Korea for providing the data set and in-depth advices related to the on-demand
food delivery industry and AI algorithms that are commonly applied in companies in this field.

References
Amit, K. 2021. "The Challenge with Gig Economy." The Economic Times.
Bokányi, E., and Hannák, A. 2020. "Understanding Inequalities in Ride-Hailing Services through
Simulations," Scientific reports (10:1), pp. 1-11.
Burton, J. W., Stein, M. K., and Jensen, T. B. 2020. "A Systematic Review of Algorithm Aversion in
Augmented Decision Making," Journal of Behavioral Decision Making (33:2), pp. 220-239.
Chui, M. 2017. "Artificial Intelligence the Next Digital Frontier," McKinsey and Company Global Institute
(47:3.6).
Dietvorst, B. J., Simmons, J. P., and Massey, C. 2015. "Algorithm Aversion: People Erroneously Avoid
Algorithms after Seeing Them Err," Journal of Experimental Psychology: General (144:1), p. 114.
Frank, M. R., Autor, D., Bessen, J. E., Brynjolfsson, E., Cebrian, M., Deming, D. J., Feldman, M., Groh, M.,
Lobo, J., and Moro, E. 2019. "Toward Understanding the Impact of Artificial Intelligence on Labor,"
Proceedings of the National Academy of Sciences (116:14), pp. 6531-6539.
Graetz, G., and Michaels, G. 2018. "Robots at Work," Review of Economics and Statistics (100:5), pp. 753-
768.
Kanazawa, K., Kawaguchi, D., Shigeoka, H., and Watanabe, Y. 2021. "Ai, Skill, and Productivity: The Case
of Taxi Drivers," Working Paper.
Luo, X., Qin, M. S., Fang, Z., and Qu, Z. 2021. "Artificial Intelligence Coaches for Sales Agents: Caveats and
Solutions," Journal of Marketing (85:2), pp. 14-32.
Seamans, R., and Raj, M. 2018. "Ai, Labor, Productivity and the Need for Firm-Level Data," National
Bureau of Economic Research.
Wood, A. J., Graham, M., Lehdonvirta, V., and Hjorth, I. 2019. "Good Gig, Bad Gig: Autonomy and
Algorithmic Control in the Global Gig Economy," Work, Employment and Society (33:1), pp. 56-75.
Zanker, C., Moll, C., Jager, A., and Som, O. 2016. "Analysis of the Impact of Robotic Systems on
Employment in the European Union," European Comission (EC).

Forty-Second International Conference on Information Systems, Austin 2021

9