Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71

Contents lists available at ScienceDirect

Robotics and Computer-Integrated Manufacturing

journal homepage: www.elsevier.com/locate/rcim

Online rescheduling of multiple picking agents for warehouse management

Jose I.U. Rubrico a, Toshimitu Higashi b, Hirofumi Tamura c, Jun Ota d,n
a
Department of Precision Engineering, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
b
Murata Machinery, Ltd., 2 Nakajima, Hashizume, Inuyama-shi, Aichi 484-0076, Japan
c
Murata Systems, Ltd., 3 Minamiochiai-cho, Kisshoin, Minami-ku, Kyoto 601-8326, Japan
d
Research into Artifacts, Center for Engineering (RACE), The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba 270-8568, Japan

a r t i c l e in f o a b s t r a c t

Article history: In this paper, we present a solution for a dynamic rescheduling problem involving new orders arriving
Received 18 January 2008 randomly while static orders have been given in advance in warehouse environments. We propose two
Received in revised form variations of an incremental static scheduling scheme: one based on the steepest descent insertion,
3 June 2010
called OR1, and the other, on multistage rescheduling, called OR2. Both techniques are enhanced by a
Accepted 3 June 2010
local search procedure specifically designed for the problem at hand. We also implemented several
existing online algorithms to our problem for evaluative purposes. Extensive statistical experiments
Keywords: based on real picking data indicate that the proposed methodologies are competitive with existing
Dynamic scheduling online schedulers and show that load-balancing algorithms, such as OR1, yield the best results on the
Warehouse management
average and that OR2 is effective in reducing the picking time when dynamism is low to moderate.
Online algorithm
& 2010 Elsevier Ltd. All rights reserved.

1. Introduction not known beforehand and arrive at random are addressed in

The problem of picking customer orders from warehouse
storage to the packing/delivery area, i.e., order picking, is one of
the most important activities in a warehouse due to its high
operational cost. Studies have shown that order picking can
amount to 60% of the total costs of warehouse operations [1].
Traditionally, due to the high percentage of travel time in an order
picker’s activity [2], order picking has been treated as a distance-
minimization problem. Optimization of a single picking route may
be found in Refs. [3–7], among others.
For multiple routes and order pickers, exact methods are
presented in Refs. [8,9], while heuristics for the cluster-then-route
approach are investigated in Ref. [10]. In a series of works, Rubrico
et al. [11–14] solve a more complicated form of the order picking
problem with multiple pickers (agents), in which each agent may
be assigned more than one trips (routes) to accomplish and delays
due to congestion or queuing are considered. The objectives are to
minimize an agent’s idle time by minimizing travel distance and
delays as well as maintain the balance of load among the agents.
They adapt a hierarchical decomposition and solve each subdivi-
sion using heuristics to achieve good schedules quickly for
realistically sized problems.
The works reported above address a static order picking
problem in which all orders to be picked are known beforehand.
Purely online versions of a picking problem in which all orders are
n
Corresponding author. Tel.: +81 4 7136 4252; fax: +81 4 7136 4242.
E-mail address: ota@race.u-tokyo.ac.jp (J. Ota).
Refs. [15,16], where stochastic distributions are assumed for the
order parameters and the problem is solved using methods
related to the queuing theory. The online picking problem can be
seen as an instance of the general online scheduling problem, in
which the agents retrieving the orders correspond to the
machines that process the jobs. A description of the online
scheduling problem and its variants, as well as surveys of eminent
online algorithms, may be found in Refs. [17–20]. Online
scheduling on uniform machines (i.e., a given job may be
processed on any machine at the same cost) has been relatively
well studied [21–25]. However, such a straightforward represen-
tation cannot be directly applied to the online picking problem
because the cost incurred by the agent as a result of a random order
will depend on the agent’s current load and is further complicated
by the occurrence of delays. A more appropriate representation of
the online picking problem is the scheduling of unrelated machines,
in which the cost of processing a given job may differ arbitrarily
from one machine to another. Literature on this difficult variation of
the online scheduling problem remains relatively sparse [26,27]. In
one study, Aspnes et al. [26] investigate the characteristics of a post-
greedy variation of Graham’s algorithm (originally intended for
scheduling on uniform machines [21]) and propose a procedure that
attains the best possible worst-case behavior (according to compe-
titive analysis [28,29]) for the online scheduling of jobs on unrelated
machines.
Another problem closely related to the online picking problem
is the dynamic vehicle routing problem (DVRP). Typical dynamic
events include: (a) customer additions/deletions, (b) changing-path
costs (e.g., road traffic status), and (c) contingencies, such as

0736-5845/$ - see front matter & 2010 Elsevier Ltd. All rights reserved.
doi:10.1016/j.rcim.2010.06.011
 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71
vehicle breakdowns and recovery. These dynamic parameters can
cause re-planning of routes and rescheduling of vehicles as well as
the addition of vehicles for dispatching to meet strict due dates.
Their correspondence with the online picking problem is
apparent, and, in it, the addition of customers corresponds to
the random arrival of new orders. A survey of the classifications,
variants, and methodologies for the DVRP may be found in Ref.
[30]. In Ref. [31], a parallel tabu search is used incrementally to
improve the solution between dynamic events (e.g., order of
arrivals). A generalization of this approach is discussed in Ref. [32]
as the multiple plan approach (MPA), which keeps multiple plans
each time an event occurs to allow for greater flexibility in the
solution choice as time progresses. MPA is a framework that is
independent of the search procedure. In the same paper, the
authors also introduce an enhancement of the MPA that includes
the forecasting capability. This approach is referred to as the
multiple scenario approach, in which the solutions kept include
predicted demands incorporated with existing ones. In Ref. [33],
a reactive architecture based on constraint programming is
proposed to deal with dynamic changes in real time, while, in
Ref. [34], a software prototype is presented that demonstrates the
capabilities of reactive vehicle routing.
In this paper, we are interested in solving the dynamic
problem of new orders arriving randomly while the static orders
are in the process of being picked. This problem is referred to here
as the online rescheduling problem because it involves modifying
the existing agent schedules in order to effectively incorporate
the new orders along with the old ones, which are yet unpicked.
The arrival of new orders is considered in this study because it is
the primary cause of dynamicity in a real picking operation. The
rescheduling problem differs from strict online scheduling, in
which all orders arrive randomly. The methods for online
scheduling are somewhat limited for the rescheduling problem
because they are unable to modify the existing schedule by
redistributing existing trips among the agents. On the other hand,
the DVRP with stochastic demands is more similar to the problem
presented in this paper. However, because of the metropolitan
scales of distances in the DVRP, a calculation time of approxi-
mately 1 min is acceptable from the methodologies reported in
the preceding paragraphs. Such times are too slow for the picking
problem and will severely delay operations. Thus, there is a need
for faster rescheduling methods for our problem which, none-
theless, yield good and robust schedules.
The objective of the study is to investigate the performance of
two variants of an incremental static scheduler that resolves the
total schedule every time a new order arrives by comparing it to
existing online schedulers in the literature and in general practice.
The incremental static framework is adapted because of its speed
and low resource overhead as compared with more elaborate
methods, such as those in Refs. [31,32].
ORDERS
PRODUCT SHELF # QTY
AA 1 10
AG 7 52 WPP
BX 50 60
: : : SOLVER
: : : :
EF 110 5
EZ 130 45
: : : :
Fig. 1. Input and output of the picking problem.
63
The rest of the paper proceeds as follows. In Section 2,
the online rescheduling problem is described. In Section 3, the
proposed and reference methodologies are described. In Section 4,
results from evaluative experiments are presented, and Section 5
is the conclusion.
2. Problem
The online rescheduling problem is a dynamic extension of the
warehouse picking problem (WPP) preliminarily described in
Ref. [11]. In the static version, a set of customer orders must be
retrieved from warehouse storage shelves by a group of picking
agents and transferred to a common packing shed. In the online
problem, new orders randomly arrive and are added to the list of
orders to be picked even while the static orders are being
retrieved. Fig. 1 illustrates an input (order list) and output (agent
picking schedule) example. For the static problem, the order list
remains unchanged in time until picking is completed, whereas,
for the dynamic WPP (DWPP), the order list continually grows in a
random manner as time passes until some terminating condition
is satisfied. Some assumptions on the dynamic problem are
enumerated below:
 static and dynamic orders have the same priority,
 all static and dynamic orders must be picked,
 a dynamic order cannot be inserted into a trip currently being
traversed by an agent,
 only order additions are considered (i.e., no cancellations),
 a newly arrived order is composed of one type of product
only.
2.1. Stochastic orders
The parameters that characterize the random orders are (a)
arrival rate l (orders per unit time), (b) product information, i.e.,
storage quantity prand and required quantity drand, and (c) total
item ratio with static orders, g ¼Drand/Dstatic, where Drand is the
total quantity of random orders picked and Dstatic the total
number of static orders.
The arrival of new orders is taken to be exponentially
distributed. A transformation can be made to express the inter-
arrival time in terms of the uniform distribution given in (1)
below. The term U denotes a uniform distribution of random
variates within interval (0,1), while tarr is the inter-arrival time,
i.e., the time interval between order arrivals:
lnðUÞ
tarr ¼ ð1Þ
l
AGENT SCHEDULE
AGENT SHELF SEQUENCE
A1 7,1,4,0; 2,13,0;
40;43;47,0; 8,0;
A2 6,9,7,0; 21,23,5,0;
60;72;75,0; 91,0;
: 48,49,50, 0; 100,0;
A8 112,115,110,0;
14,20,40,7,0; 3,0;
64 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71
As each new order arrives, the quantity of a product that it
requires is assumed to be uniformly distributed within the range
[1,Q], where Q is the capacity of an agent. Similarly, the location,
represented by the storage shelf, is also uniformly distributed in
the warehouse.
The g parameter is generally a characteristic of a specific
warehouse and may change with market seasons and other
factors. In some warehouses, this parameter may be used as a
decision variable (e.g., how many picking agents to use). For the
industry partner of this research, a common value is g ¼0.5,
indicating that almost a third of the products picked from their
warehouses are stochastic in nature. In (1) above, if tarr is taken to
be the total picking time horizon T, then tarrl ¼Tl ¼nx, where n is
the total number of orders picked, x is the ratio of random orders
to the total orders picked, and, hence, nx is the number of random
orders that arrived after time T elapsed. Note that varying either g
or arrival rate l affects the dynamicity of the system.
2.2. Objective and time constraint
The objective of the online rescheduling problem is to
minimize the makespan, i.e., the picking time, of the total
schedule. This makespan is necessarily dynamic because the
schedule does not remain fixed but is constantly modified as new
orders are added. Due to the stochastic nature of the orders, only
partial information is available regarding the total input, and a
global solution to the dynamic problem is not possible. An
alternative objective, then, is to optimize the schedule makespan
each time a new order arrives, subject to computational time
restrictions. Although this does not guarantee the arrival at a
global optimum, it does assure that a good schedule is used every
time a new order arrives.
Online rescheduling is to be done in real time, i.e., computa-
tional time must be limited to several seconds at most. In the
worst-case, an agent that has just finished loading at the shed will
wait for its new trip assignment for a time not less than that
required to recalculate the schedule. Thus, it is important to keep
the effect of this delay low to minimize on warehouse throughput.
2.3. Schedule regeneration
Fig. 2 illustrates the interaction between the static and
dynamic aspects of the picking problem. Initially, the offline
scheduler supplies the picking agents with a schedule derived
from the static orders. As new orders arrive, the online scheduler
modifies the current schedule to incorporate the new additions
and output the changes to the agents. This continues until new
orders cease to arrive. Here, the modifiable elements of an agent’s
schedule are taken to mean all the trips that are not currently
being traversed or have not yet been traversed. In contrast with
dispatching policies that are basically open-loop, the online
scheduler uses a feedback mechanism in which the state of the
current schedule partly determines the recalculated schedule.
Fig. 2. Static and dynamic aspects of the WPP.
3. Methodology
3.1. Incremental static rescheduling
The proposed online schedulers are presented here. Because of
strict time considerations, the schedulers are designed to be fast
without sacrificing effectiveness. A global rescheduling strategy is
adopted in which a complete schedule is recalculated each time
the input changes. An advantage of this approach is that fast and
efficient heuristics can be used to derive good schedules for the
updated set of orders in real time.
3.1.1. Steepest descent insertion
Fig. 3 shows the flowchart of the proposed Online Rescheduler
1 (OR1). Assuming that picking is still allowed in the warehouse,
the scheduler waits for the arrival of a random order. When there
is such an order, it is immediately routed using a VRP heuristic
(e.g., savings heuristic [35]). The resulting trip TN is then assigned
to the agent that is estimated to have the shortest makespan. A set
of untraversed trips, ST, is then created. TN is not included in ST. If
the set is empty, then the schedule is left as it is. However, if the
set is not empty, then it means that it is still possible for the
schedule to be improved by relocating demands. A fast local
search heuristic is used to implement the improvements.
Before elaborating more on OR1, the basic operators and local
search procedure used to minimize the schedule makespan are
first presented.
3.1.1.1. Basic operators. Three types of demand relocation opera-
tors are considered. A demand refers to the items that need to be
picked from a given shelf location; hence, a demand relocation
refers to the transfer or exchange of demands between agents or
within an agent (i.e., between trips of an agent), as illustrated in
Fig. 4. Here, each segment corresponds to a trip (i.e., a picking
START
Y Random N
order?
Create a trip TN containing
the new order
Assign TN to agent with
minimum makespan
Create a set of
untraversed trips ST
Y ST N
empty?
Improve schedule using
local search
Fig. 3. Diagram for OR1 procedure.
 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71 65

local relocation a1 : 0 10 (60) 0 14(60)

a1 : 0 10 (25) 73 (20) 0 131 (40) 1 (20) a2 : 0 101 (60)

non-local relocations Splitting for better

balance
a2 : 0 154 (50) 0 18 (30) 89 (5)

Fig. 4. Agent schedule and demand relocation.

a1 : 0 10 (60) 0 14(30)

cycle subject to an agent’s capacity). The numbers correspond to

the shelf identifications (0 designates the shed), while those in
a2 : 0 101 (60) 0 14(30)

parentheses refer to the number of items to be picked from the Fig. 5. Optimal non-splitting solution vs. the splitting solution.
corresponding shelf.
Let s1 be a shelf that is visited by agent a1, and let d1 be the
demand to be picked by a1 from s1. A demand transfer (M1) is a BEGIN
move that transfers d1 items of s1 from a1 to another agent, a2. A current iteration
new trip is constructed to serve s1 and then assigned to a2. Thus,
a2 will have one extra trip after the transfer. Note that a demand
transfer is allowed only if a1 has at least one remaining shelf to Determine operator and its
visit after the demand of s1 is removed from it. inputs a1, d1, a2, (d2)
The next operator is the demand exchange (M2), wherein d1
items from shelf s1 currently assigned to agent a1 are relocated to Operators
another agent, a2. Similarly, d2 items from shelf s2 currently assigned Y Demand N
to a2 are relocated to a1. After the exchange, a1 and a2 will each have exchange?
one extra trip containing the demands of s1 and s2, respectively.
The splitting demand transfer (M3) is an extension of the M1
operator that allows partial relocation of a shelf’s demands from
one agent to another provided that such a transfer improves the Remove d2 from a2 and Remove d1 from a1 and
add to a1 as a direct trip add to a2 as a direct trip
balance of the agents’ makespans. Fig. 5 illustrates such an
instance.
For any shelf s with total demand d currently assigned to agent
a1, we can estimate the number of items dn of s to be moved from Solve a VRP with Solve a VRP with
a1 to another agent, a2, so that their makespans are balanced: the nodes being the nodes being
  the shelves of a1 the shelves of a2
ðMKa1 MKa2 ÞðTs Þ
d* ¼ ð2Þ
2TLOAD Use a scheduling procedure,
NoLQ 2 to minimize delays of
MKa1 and MKa2 are the makespans of a1 and a2, respectively, the resulting schedule
before demand dn is relocated, Ts is the travel time for the new trip
assigned to a2 containing dn items of s, and TLOAD is the loading Local Search
time for a single item. In the event that the agents’ makespans
cannot be balanced by a partial relocation of p, the value of dn will Decide whether or not to
accept new schedule
be less than or equal to zero. On the other hand, if dn is greater
than or equal to d, then more than d items are needed to balance
the agents’ makespans.
END
3.1.1.2. Local search procedure. In previous studies, the introduc- current iteration
tion of direct trips serving the relocated demands may result in (a) Fig. 6. Local search used in conjunction with operators.
suboptimal travel distance for each of the affected agents, re-
sulting in longer makespans and (b) large delays caused by
loading queues that may occur after modifying the schedule un- agent a1 to agent a2. The VRP representation of each agent’s
checked. The local search (LS) procedure (Fig. 6) is applied after pallets changes accordingly from the original (Step 1) to the
every basic move operation to minimize the undesirable effects intermediate state (Step 2). The final frame (Step 3) shows the
mentioned. This figure corresponds to the rectangle ‘‘Improve improvement trips for each agent after the VRP heuristics have
schedule using local search’’ in Fig. 3. been applied.
First, the problem is to determine how to best incorporate the After the trips of the agents involved in the demand transfer/
transferred demand in the schedule of the receiving agent. This exchange have been improved, the next step is to determine
problem is non-trivial and involves solving a VRP for every whether the new schedule has queuing delays. The NoLQ2
demand transfer. Because such a task needs to be repeated procedure shown in Figs. 6 and 11, diagramed as Fig. 8, is
numerous times in the search, a fast heuristic is employed. The applied to minimize the reported delays. In the figure,
DLGIA procedure in Ref. [36] is utilized for this purpose, although
any other ‘‘good’’ VRP improvement procedure will suffice. Fig. 7
explicitly illustrates the demand insertion procedure utilizing the  numLQ—number of queues in the current schedule,
VRP procedure. Here, the demand from shelf 13 is moved from  maxLQ—maximum queue delay in the current schedule,
66 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71

Step 1. Original State Step 2. Move Operator

a1 Trips a2 Trips a1 Trips a2 Trips
4 4

7 10 8 9 7 10 8 9

move
13 13

23 23

direct
0 0 0
trip 0
Q = 60 Q = 60 Q = 60 Q = 60

a1 : 0 10 (25) 23 (20) 0 7 (40) 13 (20) a1 : 0 10 (25) 23 (20) 0 7 (40)

direct trip

a2 : 0 4 (50) 0 8 (30) 9 (5) move a2 : 0 4 (50) 0 8 (30) 9 (5) 0 13 (20)

Step 3. VRP Solutions

a1 Trips a2 Trips
4

7 10 8 9

13

23

0 Q = 60 0 Q = 60

a1 : 0 23 (20) 0 7 (40) 10 (25)

regroupings

a2 : 0 4 (50) 8 (30) 0 9 (5) 13 (20)

Fig. 7. Illustrating the VRP procedure as part of the local search.

 mkmaxNumLQ—number of queues for the agent with the
maximum makespan, and
 curLQ—the current queue being minimized.
The parameters above are estimated using a fast, variable
time-step schedule simulator embedded in the improvement
loop. The simulator is run each time the schedule is modified to
obtain the estimates.
The RESEQUENCE procedure will reassign the dispatch se-
quence of a trip assigned to an agent if doing so minimizes or
eliminates loading queue delays, as shown in Fig. 9, while the
INVERT procedures reverse the traversal sequence of a trip if this
minimizes or eliminates loading queues, as shown in Fig. 10.
In the following figures, tLQ refers to the trip(s) containing the
products at the shelf pLQ that cause the queuing delay. The
function f1() returns the makespan.
3.1.2. Online multistage rescheduling
The second online rescheduler is referred to as OR2. For this
variant, whenever a new order arrives, a VRP improvement
procedure is applied to all currently unserved routes plus the
direct trip serving the new order. The resulting routes are then
reassigned to the agents using the load-balancing techniques
presented in Ref. [11] and a queue-minimization procedure
similar to that described in Section 3.1.1. This procedure
minimizes the makespan by reducing the idle time associated
with the agents, i.e., the travel time and queuing delays. The
logical diagram of the OR2 and its listing are given in Fig. 11.
3.2. Other algorithms for comparison
Among the variants of the online scheduling problem, the one
most similar to our problem is that of scheduling unrelated
machines. This variation is the most general and involves
machines with arbitrary processing speeds.
As far as we know, there are only two algorithms on the said
scheduling problem that have been formally investigated [26]. We
implement both as a basis of comparison for the proposed
methods. They are selected because they solve a similar problem
to ours and are considerably fast.
The first one is the G66 Listing algorithm, which is an
extension of Graham’s list scheduling [21] to the DWPP. The
main idea of this algorithm is to balance the load among the
agents by assigning a new order to the agent that will increase its
makespan the least if the new order is added to it.
 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71 67

The second one is an extension of the Aspnes’93 algorithm [4]
to the DWPP. This algorithm tends to create a load imbalance if
the agents’ makespans are roughly the same instead of always
maintaining the load balance.
3.3. Reference online scheduler
The reference is an online extension to a widely practiced
procedure to schedule picking agents. Initially, trips are generated
according to the S-shaped heuristic [21] and then assigned
sequentially to each agent. An agent is assigned a new trip as
soon as it reaches the shed and becomes idle. This goes on until all
the orders are picked.
The logic of the online reference scheduler is diagramed in the
flowchart in Fig. 12. Assuming that picking is still allowed in the
warehouse, the scheduler continually checks for the presence of
an idle agent in the shed. If there is such an agent, then it is
assigned the next available trip generated from the offline orders.
However, if all offline trips have already been assigned, then the
warehouse order queue is checked for pending (i.e., random)
orders. If there are any, the orders are routed, and the resulting
trip is assigned to the idle agent. A waiting loop is entered when
there are no idle agents in the shed or no pending orders to be
routed.
4. Experiments

STA RT 4.1. Simulation setup

Agent schedu le a fter The experiments are done on a set of real data instances. A
mo ve ope rator a nd
NoLQ 2 V RP heu ristics are appl ied summary of the base static instances is given in Table 1. For every
shelf distribution, there are two different item densities (i.e., the
average number of items picked per active shelf) with one density
Y N higher (‘‘a’’ suffix) than the other (‘‘o’’ suffix). The agent capacity Q
n umLQ > 0
is taken to be 60 items. The instances generally have irregular
Y N shelf distributions with some small clusters in a number of aisles.
mkmaxNu mLQ > 0
The warehouse is large with 1000 storage shelves. The aisles of
the warehouse are wide enough so that two picking agents can
cu rLQ = mkmaxNu mLQ cu rLQ = maxL Q
pick side by side with room to spare for maneuvers. Such
characteristics reflect the kind of setup used in the real system
RESEQ UENCE
considered here.
Initial solutions are first generated for the base instances. Then,
while the orders are being picked, new orders are allowed to
Y N
IMPRV ? arrive randomly. For each static data, 10 online instances are
generated (i.e., using different random number generator seeds).
INVERT In the experiments, g ¼ 0.2, 0.5, and 0.8, and m ¼8 agents to
correspond to the common situations in the warehouse of the
Y N industry partner. To investigate the sensitivity of the online
IMPRV? reschedulers to the inter-arrival times of the random orders, the
experiments are run on l ¼0.0025, 0.005, 0.01, 0.025, 0.05, and
0.1. Thus, for each online algorithm tested, 6  10  3  6¼1080
Agen t schedu le
after delay simulations are done, resulting in 5400 total simulations run for
management the five procedures evaluated.
Except for the reference scheduler, all other schedulers use the
STOP same initial static schedule (for a given base instance), in which
each agent is already assigned the complete list of trips that it
Fig. 8. Diagram of the NoLQ2 procedure. must complete at the start of the picking process. In contrast to

Original Trip Sequence of aLQ (Load Queues not shown)

t1 tLQ t6 t1 0
aLQ

Reinsertions of tLQ in aLQ (Load Queues not shown)

tLQ t1 t6 t1 0
aLQ

t1 t6 tLQ t10
aLQ

t1 t6 tLQ t10
aLQ

t1 t6 t10 tLQ
aLQ

Fig. 9. Trip reinsertion, RESEQUENCE.

68 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71

Trips of agents sharing a pallet, pLQ, causing a load queue

tLQ 1
t1 t6 t1 0
aLQ 1 pLQ p12 p14

tLQ 2 overlap
t2 t8 t3
aLQ 2 p15 pLQ

Possible Inversions of tLQs to avoid loading queues

tLQ 1
t1 t6 t1 0
aLQ 1 p14 p12 pLQ

tLQ 2
t2 t8 t3
aLQ 2 p15 pLQ

tLQ 1
t1 t6 t1 0
aLQ 1 pLQ p12 p14

tLQ 2
t2 t8 t3
aLQ 2 pLQ p15

Fig. 10. Trip inversion, INVERT.

START START

Y agent at N
Y Random N shed?
Order?

Y Offline N
create a trip TN containing
Trips?
the new order

Y Assign next offline

trip in sequence to Y Random N
Y N agent Orders?
Untraversed
Routes?

Route orders using

S-Shape
“Improve routes” Assign TN to agent with
procedure minimum makespan
Assign new
trip to agent
“Reassign routes to
agents” procedure
Fig. 12. Online reference scheduler.
“NoLQ2(s)”
procedure
Table 1
Characteristics of the static base instances, Q¼ 60.
Fig. 11. Diagram for the OR2 procedure.
Instance Items Shelves Shelf distribution Average item density

the S-shaped heuristic used by the reference, all other schedulers MDAT1o 779 75 Irregular 0.1731
generate their trips using the parallel savings heuristic [37]. MDAT1a 1136 75 Irregular 0.2524
MDAT2o 758 45 Irregular 0.2807
MDAT2a 960 45 Irregular 0.3556
MDAT3o 234 30 Irregular 0.13
4.2. Results MDAT3a 480 30 Irregular 0.2667

Comparative results of the experiments are presented here.

The evaluation is done on the basis of a makespan. In the
discussions that follow, the term ‘‘scheduler’’ is used in place of
‘‘rescheduler’’ for convenience.
Fig. 13 shows the makespans obtained using the schedulers
tested. The values correspond to the average taken over 60 data
instances (6 base instances  10 online instances per base
instance) for a given l and g. For a fixed g, uncertainty increases
as l decreases because random orders will arrive more slowly and
the system degenerates into a first-come-first-served (FCFS)
 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71 69

queue in which planning becomes meaningless. As l increases,
random orders arrive quickly enough so that they can be
rescheduled with most of the yet unpicked static orders, thus
reducing the uncertainty in the system. On the other hand, for a
fixed l, dynamicity increases with g since it would mean an
increased ratio of random orders.
The percent improvement of OR1, OR2, AS93, and G66 against
the reference (REF) scheduler is tabulated in Table 2. A negative
improvement indicates that the solution of the corresponding
scheduler is worse than that of the reference.
The following observations can be made from the makespan
results:
 All schedulers tend to perform equivalently as dynamicity
increases (i.e., small l and large g). This is apparent in the way
the makespan values converge as l decreases.
 OR1 performs significantly better than the reference except in
the most dynamic cases tested, in which its performance is
equivalent to the reference. Its improvements tend to increase
as dynamism decreases, as expected from an incremental
static scheduler. This results in the best average improvement
of 8.31% over the reference.
 OR2 performs significantly better than the reference when
dynamism is low to moderate. However, when uncertainty is
high, it tends to perform worse than the reference. This results in
both the highest (15.53%) and the lowest ( 4.96%) improve-
ment. This suggests that distance optimization is more effective
in minimizing the makespan when sufficient planning is
applicable (i.e., many orders are still available to be rescheduled
jointly). When the system becomes more dynamic, random
orders arrive piecemeal in an FCFS fashion, and minimizing the
travel time does not necessarily minimize the picking time. OR2
shows an average improvement of 6.17% over the reference.

3100
2900
2700 REF
2500
Seconds

OR1
2300 OR2
2100 AS93
1900 G66
1700
1500
0.0025 0.005 0.01 0.025 0.05 0.1
Lambda

5300
4800
4300 REF
Seconds

3800 OR1
OR2
3300
AS93
2800 G66
2300
1800
0.0025 0.005 0.01 0.025 0.05 0.1
Lambda

7500
7000
6500
6000 REF
5500
Seconds

OR1
5000
OR2
4500
4000 AS93
3500 G66
3000
2500
2000
0.0025 0.005 0.01 0.025 0.05 0.1

Lambda

Fig. 13. Averaged makespans for indicated schedulers. (a) g ¼ 0.2, (b) g ¼ 0.5, and (c) g ¼ 0.8.
70 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71

Table 2 best results on the average. They are consistent, either matching
% Improvement of indicated schedulers over the reference. the performance of the reference (within 1%) or improving it
significantly (by 6% or more). On the other hand, the tests show
Scheduler Arrival rate k (orders per unit time)
and c that algorithms possessing desirable worst-case margins (AS93)
0.0025 0.005 0.01 0.025 0.05 0.1 AVE do not necessarily perform well on the average. It is also evident
that incremental distance minimization (OR2) is effective in
OR1 0.2 1.59 6.01 12.16 14.29 14.04 13.14 10.20 reducing the picking time when dynamism is low to moderate.
0.5 0.46 1.44 8.33 13.15 12.07 13.77 8.20
0.8  0.08 1.41 7.64 9.94 10.32 9.81 6.51
However, it performs badly when dynamism is high.
These results show that the proposed online reschedulers are
OR2 0.2  1.40  1.88 7.78 11.52 12.12 12.74 6.81 competitive against existing online methods when uncertainty in
0.5  4.31  4.97 3.24 13.70 14.73 15.53 6.32 the system is low to moderate. These are the levels of dynamism
0.8  3.46  4.96 3.18 11.57 12.81 13.16 5.38
often encountered in real picking systems that tend to restrict the
AS93 0.2  0.05  1.35  0.83  1.55  1.55  1.55  1.15 incorporation of random orders with existing ones for account-
0.5 0.14 0.55 5.90 4.95 5.13 5.13 3.63 ability and due-date concerns. Thus, OR1 and OR2 can provide
0.8  0.12 1.11 5.03 4.03 4.00 4.00 3.01 significant improvements in real picking systems that have a mix
of static and stochastic inputs.
G66 0.2 0.83 2.53 6.49 6.71 6.44 6.74 4.96
0.5 0.20 2.10 7.12 8.98 9.13 8.98 6.08
There are several limitations of the proposed algorithms. One
0.8  0.08 1.44 7.35 6.90 6.61 6.82 4.84 is that we assume the situation in which only a new order is
coming. However, in real working environments, many other
disturbances may occur, such as a disorder of some apparatus or
 AS93 performs as well as the reference when dynamism is high delay in the completion of a certain operation. We need to take
and provides slight to moderate improvements as the system such effects into account. Another important issue is that agents
becomes more static. Its average improvement is 1.83%. are heterogeneous and can change dynamically in real environ-
 The G66 algorithm performs as well as the reference when ments. In that case, it is not easy to measure the performance of
dynamism is high. It performs significantly better when each agent precisely, and it may introduce bad performance in
dynamism is decreased. This is interesting in that the greedy rescheduling. We need to deal with the problem as a future work.
algorithm theoretically has a much larger worst-case margin
than the AS93 (optimal worst-case margin). On the average, it
shows improvement of 5.29% over the reference. References
 All recalculation times are real time and do not exceed 5 s.
[1] Drury J. Towards more efficient order picking. IMM monograph no. 1, The
Institute of Materials Management, Cranfield, UK: 1988.
[2] Tompkins JA, et al. Facilities planning. New York: John Wiley & Sons; 1996.
The first observation results from the characteristic breakdown [3] Ratliff HD, Rosenthal AS. Order-picking in a rectangular warehouse: a
of any scheduler into an FCFS policy when the remaining random solvable case for the traveling salesman problem. Oper Res 1983;31(3):
507–21.
inputs continue to arrive long after all static and early random [4] Roodbergen KJ, De Koster R. Routing methods for warehouses with multiple
orders have been picked. In this state, the makespans of different cross-aisles. Int J Prod Res 2001;39(9):1865–83.
schedules level out as the agents remain idle and simply wait for [5] Roodbergen KJ, De Koster R. Routing order-pickers in a warehouse with a
middle aisle. Eur J Oper Res 2001;133:32–43.
the arrival of the next order. The second and third observations
[6] De Koster MB, Van Der Poort ES, Wolters M. Efficient orderbatching methods
are consistent with the fact that, when dynamism is low to in a warehouse. Int J Prod Res 1999;37(7):1479–504.
moderate, redistributing trips among agents is beneficial because [7] Hall RW. Distance approximations for routing manual pickers in a warehouse.
of the more global consideration in improving the makespan than IIE Trans 1993;25(4):76–87.
[8] Gademann AJRM, Van Den Berg JP, Van Der Hoff HH. An order batching
the restricted approach adopted by the reference and online algorithm for wave picking in a parallel-aisle warehouse. IIE Trans
schedulers (G66, AS93). It also indicates that minimizing the 2001;33(5):385–98.
average makespan by reducing the idle travel time (OR2) does not [9] Gademann AJRM, van de Velde S. Order batching to minimize total travel
time in a parallel-aisle warehouse. IIE Trans 2005;37(1):63–75.
necessarily minimize the maximum makespan when randomness [10] Rosenwein MB. An application of cluster analysis to the problem of locating
is high because there will be fewer available trips that OR2 can items within a warehouse. IIE Trans 1994;26(1):101–3.
reroute. On the other hand, OR1 reduces to a load-balancing [11] Rubrico JIU, Ota J, Higashi T, Tamura H, Akiyoshi M. Route generation for
warehouse management using fast heuristics. In: IEEE/RSJ international
scheduler under such circumstances and performs well, not conference proceedings on intelligent robots and systems. Sendai: 2004,
unlike G66, which also strives to achieve makespan balance. The p. 2093–98.
AS93 is unable to remain competitive with the other schedulers [12] Rubrico JIU, Ota J, Higashi T, Tamura H. Route assignment heuristics for
picking products in a warehouse, human and artificial intelligence systems:
because it sacrifices balance to minimize the worst-case perfor- from control to autonomy. In: Murase K, Jain L, Sekiyama K, Asakura T,
mance. Thus, unless a degenerative instance is being scheduled, it editors. Proceedings of the fourth international symposium on human and
often underperforms. artificial intelligence systems. Advanced Knowledge International; 2004.
p. 341–46.
[13] Rubrico JIU, Ota J, Higashi T, Tamura H, Akiyoshi M. Multi-agent scheduling in
a warehouse. In: Proceedings of the international conference on instrumen-
5. Conclusion tation, control and information technology, SICE 2005, Okayama, 2005.
p. 3243–47.
[14] Rubrico JIU, Ota J, Higashi T, Tamura H, Scheduling multiple agents for
In this paper, we address a problem of online rescheduling in picking products in a warehouse. In: Proceedings of international conference
which new orders arrive randomly while static orders are still on robotics and automation, ICRA 2006, Orlando. 2006, p. 1438–43.
being picked. Two real time incremental static reschedulers are [15] Chew EP, Tang LC. Travel time analysis for general item location assignment
in a rectangular warehouse. Eur J Oper Res 1999;112(3):582–97.
proposed: OR1, which is based on a steepest descent insertion
[16] Le-Duc T. Design and control of efficient order pickling processes. PhD thesis.
scheme, and OR2, which is based on a multistage optimization The Netherlands: Erasmus University Rotterdam; 2005.
procedure (distance minimization, load-balancing, and queuing [17] Albers S, Competitive online algorithms, BRICS lecture series LS96-2,
delay minimization). Department of Computer Science, University of Aarhus, 1996.
[18] Sgall J. In: Fiat A, Woeginger G, editors. Online scheduling. In: Developments
Extensive statistical experiments on real data instances reveal from a June 1996 seminar on online algorithms: the state of the art, 1442.
that load-balancing algorithms, such as OR1 and G66, yield the Springer LNCS; 1998. p. 196–231.
 J.I.U. Rubrico et al. / Robotics and Computer-Integrated Manufacturing 27 (2011) 62–71
[19] Azar Y. In: Fiat A, Woeginger G, editors. On-line load balancing, developments
from a June 1996 seminar on online algorithms: the state of the art, 1442.
Springer LNCS; 1998. p. 178–95.
[20] Winter T, Zimmermann UT. Discrete online and real-time optimization,
In: Proceedings of the 15th IFIP world computer congress, Budapest/Vienna,
1998, p. 31–48.
[21] Graham RL. Bounds for certain multi-processing anomalies. Bell Syst Tech J
1966;45:1563–81.
[22] Bartal Y, Karloff H, Rabani Y. A better lower bound for on-line scheduling. Inf
Process Lett 1994;50(3):113–6.
[23] Bartal Y, Fiat A, Karloff H, Vohra R. New algorithms for an ancient scheduling
problem. J Comput Syst Sci 1995;51(3):359–66.
[24] Karger D, Phillips S, Tong E, A better algorithm for an ancient scheduling
problem. In: Procedings of the 5th ACM–SIAM symposium on discrete
algorithms, 1994, p. 132–40.
[25] Albers S. Better bounds for online scheduling. SIAM J Comp 1999;29:
459–73.
[26] Aspnes J, Azar Y, Fiat A, Plotkin S, Waarts O. On-line load balancing with
applications to machine scheduling and virtual circuit routing. In: Proceed-
ings of the 25th ACM annual symposium on the theory of computing, 1993,
p. 623–31.
[27] Caragiannis I. Better bounds for online load balancing on unrelated machines.
In: Proceedings of the 19th annual ACM–SIAM symposium on discrete
algorithms, 2008, p. 972–81.
71
[28] Sleator DD, Tarjan RE. Amortized efficiency of list update and paging rules.
Commun ACM 1985;28(2):202–8.
[29] Karlin A, Manasse M, Rudolph L, Sleator DD. Competitive snoopy caching.
Algorithmica 1988;3(1-4):79–119.
[30] Gendreau M, Potvin J. In: Crainic T, Laporte G, editors. Dynamic vehicle routing
and dispatching the fleet management and logistics. Kluwer; 1998. p. 115–26.
[31] Gendreau M, Guertin F, Potvin JY, Taillard E. Parallel tabu search for real-time
vehicle routing and dispatching. Trans Sci 1999;33(4):381–90.
[32] Bent R, Van Hentenryck P. Dynamic vehicle routing with stochastic requests.
In: Proceedings of the international joint conference on artificial intelligence
(IJCAI), Acapulco, Mexico, 2003, p. 1362–63.
[33] Zhu K., Ong K. A reactive method for real time dynamic vehicle routing
problems. In: Proceedings of the 12th IEEE international conference on tools
with artificial intelligence (ICTAI), Vancouver, BC, Canada, 2000, p. 176–80.
[34] Lau HC, Seah D, An efficient technique for routing of vehicles reactively.
In: Proceedings of the 8th ITSA annual meeting and exposition, Detroit,
Michigan, 1998.
[35] Clarke G, Wright JW. Scheduling of vehicles from a central depot to a number
of delivery points. Oper Res 1964;12(4):568–81.
[36] Dror M, Levy L. A vehicle routing improvement algorithm: comparison of
‘‘greedy’’ and a ‘‘matching’’ implementation for inventory routing. Comput
Oper Res 1986;13(1):33–45.
[37] Altinkemer K, Gavish B. Parallel savings based heuristic for the delivery
problem. Oper Res 1991;39(3):456–69.