Professional Documents
Culture Documents
manuscript
Rajeeva Moorthy
The Institute of Operations Research and Analytics, National University of Singapore, rajiv@nus.edu.sg
Chung-Piaw Teo
NUS Business School, The Institute of Operations Research and Analytics, National University of Singapore,
bizteocp@nus.edu.sg
Problem Definition: The rollout of the COVID-19 vaccine presents a complex challenge that involves coor-
dinating supply scheduling, capacity planning, and demand management. Vaccines are in limited supply
and often have irregular schedules, while appointment bookings significantly influence vaccine take-up and
demand. Effectively releasing appointment slots is crucial to reducing blockages in the booking system, min-
imizing waiting times for vaccination, and maximizing vaccination rates.
Methodology/Results: This study proposes an optimization framework for controlling the release and
booking of two-dose COVID-19 vaccine appointments and optimizing invitation schedules. The framework
aims to maximize vaccination rates while minimizing appointment waiting times, considering limited vaccine
supplies and uncertain demand patterns. The framework is integrated into a simulation tool that generates
vaccine take-up estimates based on optimized appointment bookings and invitation schedules, enabling the
evaluation of various vaccination rollout policies for policy guidance.
Managerial Implications: The optimization model and simulation tool developed in this study provide
support for the Singapore Ministry of Health in vaccine rollout and policy evaluations. Notably, the Singapore
government implemented a policy to double the interval between the first and second vaccine doses during
the campaign to enhance the vaccination rate. Our policy evaluation indicates that this interval-stretch
policy could save 30 days in vaccinating 4 million residents with at least one dose, compared to the non-
stretch policy. By implementing this interval-stretch policy at the appropriate time, Singapore significantly
accelerated its vaccination rate, ensuring a vaccination rate of over 75% by September 2021, just before
the Delta variant outbreak in the country. The successful implementation of this policy was one of the key
factors contributing to the low number of deaths in Singapore.
1. Introduction
Since the publication of the genetic sequence of SARS-CoV-2, the coronavirus responsible for
COVID-19, significant efforts have been dedicated to developing vaccines against this epidemic
disease and identifying priority groups for vaccination, aiming to protect lives and curb the spread of
the virus. The ability to swiftly translate vaccines into vaccinations has been pivotal in the fight to
restore normalcy. Early on, the primary bottleneck in this endeavor was the supply of vaccines, given
the global demand and the time required for production and distribution scale-up. Many countries
faced potential delays or shortages due to limited production rates and logistical constraints. On
the demand side, vaccine hesitancy resulted in not all eligible individuals coming forward to receive
vaccines. Furthermore, public attention and discussions surrounding immunization mechanisms,
potential side effects, and associated risks led to preferences for specific vaccine types or, in some
cases, a complete aversion to vaccination (Booth and Adam 2021, Dai and Song 2021). These
factors, namely vaccination hesitancy and the mismatch between vaccine supply and preference,
posed significant barriers to achieving a rapid and efficient vaccination process. Consequently,
certain vaccine types faced longer waiting times while others experienced higher rates of leftover
and expired doses. Overcoming these challenges necessitated a well-coordinated supply chain and
synchronized demand planning to achieve a high vaccination rate.
The majority of authorized COVID-19 vaccines require a two-dose regimen with specific recom-
mended intervals (e.g., 21 days for Pfizer vaccines and 28 days for Moderna). The recommended
time intervals require a careful design of the rollout policy such that individuals can complete two
doses adhering to the timing requirement. The challenge is to ensure the availability of vaccines
for the second dose given that future supply may not be sufficient. Pertaining to this unique chal-
lenge, various vaccine rollout policies were proposed and implemented in different countries. For
instance, Singapore initially adopted a conservative “holdback” policy, following the manufacturer-
recommended two-dose interval, which reserved an extra dose for every recipient of the first dose.
This policy aimed to eliminate the risk of vaccine shortages in case of supply constraints but resulted
in a slower uptake of first doses. A similar holdback policy was also initially implemented in the
United States. In an effort to accelerate the vaccination rate, the U.S. government transitioned to
a “stock-release” policy, which eliminated intentional reserves for second doses (Murray 2021). On
the other hand, the U.K. government opted for a “stretch” policy, extending the two-dose interval
from the manufacturer’s recommendation to 12 weeks (GovUK 2020). Other countries, such as
Canada, followed similar strategies after observing a surge in cases. Both the release and stretch
policies, compared to the holdback policy, expedite the administration of first doses, enabling
more individuals to acquire partial immunity sooner. However, these policies come with the risk
of delaying the administration of second doses, particularly during supply shortages. To enhance
vaccine coverage and respond effectively to the evolving situation, healthcare authorities need to
dynamically adjust their policies at different stages of disease control.
hesitancy. It is essential to explore analytical approaches that address these operational challenges
and evaluate policy performance concurrently with clinical studies. In the following, we summarize
the primary operational decisions faced by Singapore vaccination rollout implementations.
In the context of vaccination rollout, a critical decision that significantly impacts the performance
of the rollout policy is determining the number of daily first-dose and second-dose appointments
to be released in the appointment booking system, referred to as the ”daily appointment booking
limits” for each dose. For instance, under the holdback policy, the first-dose booking limit is set as
half of the available vaccines, while the release policy increases it to match the current stock level.
These two policies represent the extremes of a design spectrum, with one being overly conservative
and the other carrying higher risk. The number of the first doses administered is strategic to
the performance of a rollout policy. Specifically, the current first-dose bookings determine future
second-dose bookings with the strict dose-interval requirement, which, in turn, affects (blocks) the
capacity to accommodate future first doses. This endogeneity is difficult to address by myopic
approaches such as holdback or release policies that either hold a full reserve of the second doses
or none at all. A systematic approach that accounts for the endogeneity between first and second
doses and models it dynamically would enable better control and coordination of the vaccination
process.
Invitation Schedules and Booking Pattern. Controlling the number of eligible people in
each invitation wave is another way to manage demand for vaccinations, in addition to booking
limits. In the countries that allow walk-ins, no control over demand arrivals is imposed. It may put
a great challenge to the vaccination center’s operation when vaccination capacity is limited. Singa-
pore mostly uses invitation-only appointments to shape the demand and consumption of vaccines.
The invitation is extended to each eligible subgroup in phases and suitably staggered to avoid
appointment competition and delays. Long lead time has been shown to discourage appointment
bookings and lead to a high rate of appointment cancellation and no-shows (Green and Savin
2008), thus exacerbating vaccine hesitancy. Designing a reasonable invitation schedule that reduces
the appointment waiting time without slowing down the vaccination rate is crucial to an effective
rollout policy. Another phenomenon that adds to the complexity of the problem is that not all
invitees immediately book appointments on the nearest date available. There are time lags between
the time of receiving an invitation and the time slots selected by the invitees. Figure 1 shows the
typical booking patterns (time needed to take up a slot in the system) for three early invited groups
in Singapore. It shows booking patterns peak one to two weeks after an invitation. These factors
should be taken into account when designing invitation schedules.
We support the MOH on vaccine rollout policy design by developing an “optimization + simu-
lation” framework. For a given policy, optimization models are formulated to optimize operational
decisions, including appointment booking slot release and invitation schedule to minimize the peo-
ple’s waiting time to get the first dose appointment and maximize the vaccination rate. This is
followed by a simulation model that projects future vaccine take-ups under the optimized oper-
ations and evaluates the policy performance in terms of vaccination rate and others that matter
to the policy-maker. By comparing various performance measures simulated for policy candidates
under the optimized operations, the framework provides policy recommendations.
Figure 1 Actual booking patterns for three early priority groups in Singapore. The values on the y-axis are blinded
due to the sensitive nature of the data.
Policy Implications: Using the optimization and simulation framework in the context of Sin-
gapore, we evaluated various vaccine rollout policies, considering different levels of second-dose
reserves and two-dose intervals. As of March 2021, our framework helped us study numerous alter-
natives and we found that a shift in mid-May towards an aggressive policy of a 6 (8) week stretch
interval for Pfizer (Moderna) with a 3-week reserve would save 30 days in vaccinating 4 million
residents with at least one dose, without risk of vaccine stockout. Interestingly, Singapore did tran-
sition from a holdback policy to a stretch policy that doubled the two-dose interval in May 2021.
As a result, they achieved a 75% vaccination rate by September 2021, fortuitously just before the
arrival of the first major wave of the Delta variant in the country (see Figure 2). The course of
history may have been significantly different had this change in policy not been implemented in a
timely manner.
Figure 2 (Color online.) Color-coded curves are vaccination take-up rate (full regimen) by age group. The data is
extracted from Data.Gov.Sg; The black curve shows the daily reported COVID-19 Cases (OurWorldIn-
Data 2023). It shows that more than 75% of the population was vaccinated before the first major
COVID-19 wave of infections hit Singapore in September 2021.
2. Literature Review
Our paper makes a significant contribution to the existing body of literature that explores oper-
ational challenges in the COVID-19 vaccination process. The unique two-dose regimen of the
COVID-19 vaccines necessitates a departure from the results obtained from studies primarily
focused on single-dose vaccines (e.g., Chick et al. 2008, Arifoğlu et al. 2012, Yamin and Gavious
2013, Arifoğlu and Tang 2021). In a different context, Duijzer et al. (2018) addresses the allocation
of a general vaccine in both single and multi-dose settings.
Recent work by Oceguera-Castillo et al. (2022) offers a systematic review of COVID-19 vac-
cination research, encompassing elements such as vaccine allocation, storage, distribution, and
administration operations. Dai and Song (2021) outlines key challenges encountered in COVID-
19 vaccination operations, including the management and matching of supply and demand, and
the allocation of first and second vaccine doses. Additionally, they highlight the complexities of
appointment scheduling. In a similar vein, Zhang et al. (2021) delves into the optimization of pub-
lic policies in light of individual responses and addresses the question of vaccination prioritization
amid limited vaccine inventory. Bennouna et al. (2022) proposes the incorporation of COVID-19
case predictions into vaccine allocation optimization. Chen et al. (2020) presents a study on the
allocation of the COVID-19 vaccine among different age groups under limited supply. Innovative
solutions such as the use of drone technology for equitable vaccine distribution are explored by
Wang et al. (2023), while Karakaya and Balcik (2023) designs a national pandemic vaccination
calendar to maintain a reliable schedule in the face of uncertain supplies. A closely related study by
Mak et al. (2022) sheds light on the unique two-dose challenge and investigates its implications on
the effectiveness of vaccine rollout by comparing three existing rollout policies: holdback, release,
and stretch, under limited supply. Our paper builds on these insights by proposing a methodology
for deriving the optimal rollout policy. While Tuite et al. (2021) also compares rollout policies under
limited supply, they do not consider operational decision-making. Bai et al. (2023) explores the
allocation of limited vaccine supplies between first and second doses, focusing on the comparison of
prioritizing first-dose and second-dose usage. Calafiore et al. (2023) optimizes the administration of
first and second doses under uncertain vaccine supplies to maximize vaccine coverage. In contrast
to the aforementioned studies, our research seeks to integrate operational decision optimization,
such as daily dose administration and invitation scheduling, with vaccine rollout policy evaluation.
Our ultimate goal is to provide a comprehensive decision-support tool for policymakers.
Our paper finds considerable alignment with the expansive body of literature on immunization
scheduling and appointment scheduling. However, it specifically addresses the unique challenges
associated with the COVID-19 vaccination process, which are characterized by an urgent timeline
and a limited supply of vaccines. These challenges are generally not present in the scheduling
problems associated with other vaccines, which are typically not administered under pandemic
conditions (see, for example, Engineer et al. 2009, and the references included therein). For a
detailed review on appointment scheduling, the readers can refer to Cayirli and Veral (2003), which
presents an extensive review on appointment scheduling in outpatient services, and Gupta and
Denton (2008) that highlights various healthcare issues and challenges. The majority of works
studied in appointment scheduling literature consider a single appointment for each patient. The
subsequent repeat visits are considered new arrivals that are independent of previous visits. The
two-dose requirement feature cannot be captured. A more related paper, Yu et al. (2020), considers
the appointment scheduling for patients with a “series” of appointments. They highlight an issue
faced in such an appointment scheduling system similar to the “blocking phenomenon” we observe
in the two-dose vaccination context that the follow-up visits take up capacities to get new patients.
They optimally determine the staffing levels to maximize the healthcare system’s revenue. Given
that the staffing level that constrains the available appointment slots is on a daily basis, an MDP
model is adopted. The key difference in the context of vaccine appointment bookings is that we need
to control the waiting time and consider vaccine supplies which constraints the total appointment
bookings across periods.
The principle of booking limits is widely utilized across various applications in revenue manage-
ment, such as airline scheduling and capacity allocation (Talluri and Van Ryzin 2004), particularly
when limited capacity or supply must be apportioned across multiple demand streams. In the sce-
nario of COVID-19 vaccination, strategic allocation of the vaccine supply is crucial for both first
and second doses. This paper takes the concept of booking limits a step further by applying it to
cases where each demand necessitates two units of “products”, which must be fulfilled within a set
time interval.
3. Optimization Models
We consider a policy candidate parameterized by two parameters: (i)“safety lead time” denoted by
k, representing the number of days of reserve kept in the system for future second-dose appoint-
ments, and (ii) “two-dose stretch interval” denoted by I, representing the interval in days between
the first and second doses. These parameters determine the level of conservatism in the vaccine
rollout policy. In this session, we introduce optimization models that optimize key operational
decisions for a given policy. Specifically, for a policy with parameter (k, I), the goal is to determine
the optimal number of slots to release (i.e., booking limits) and the optimal number of people
invited (i.e., invitation schedule) into the system for appointment booking. In the next session, we
will present the simulation model so that it facilitates the performance comparison among policies
(with optimized operations) of different parameters (k, I).
T Planning horizon of the first-dose appointments (in days). r Population take-up rate of vaccine invitations.
Parameter
N Total number of eligible population. B Number of types of vaccines.
µt the expected proportion of the invited population choosing appointments on Day t.
Appointment Booking Limit Optimization
I Two-dose interval (in days). Ct Vaccination capacity on Day t.
ms Mini-stretch interval (in days). sut Amount of vaccine supplies arrive on Day t.
Parameters
k Safety lead time. slt Amount of vaccines that are expired on Day t.
Number of individuals prefer an appointment on Day t Number of individuals waiting for the first-dose appointments
Dt wt
under throughput-maximizing invitation schedule. on Day t.
Number of people having the first-dose appointment on Day t
xt The first-dose appointment booking limits on Day t. zt,t′
Decision and the second-dose appointment on Day t′ .
Variables yt The second-dose appointment booking limits on Day t. vt The buffer of expired vaccines on Day t.
Invitation Schedule Design
Iˆ Invitation interval. λ Weight in the objective function of the unfilled booking slots.
Parameters
M Number of invitations (waves) in the planning horizon. li The proportion of people that prefer vaccine type i.
Random
p˜t Random variable denoting the choice probability of an appointment on Day t.
Variable
Decision
nm Number of individuals to invite in Wave m.
Variable
Table 1 Parameters and Notations
The notations and parameters used in the model are summarized in Table 1. Let T denote the
planning horizon for the first dose (in days). Therefore, the second dose is administrated over
t = I, ..., T ′ days, where T ′ = T + I + ms . The mini-stretch, denoted by ms , offers flexibility in
the operation. In the case of a strict two-dose interval requirement, ms = 0. There is a total of N
eligible population to invite, and the take-up rate is r. Hence, a total number of rN individuals will
book the appointments. Let µt denote the expected proportion of the invited population choosing
appointments on Day t, and Dt denotes the throughput-maximizing demand arrival pattern, that
is the estimated number of individuals who prefer to book a first-dose appointment on Day t if
they are invited on Day 1, i.e., Dt = rN µt . Note that the actual booking day might not be the
same as their preferred booking day if the booking limit is not enough to accommodate everyone
sharing the same preference. In this case, they have to book the next available slots, and this time
gap amounts to waiting time in our context.
The demand arrivals (upon invitation), i.e., µt , are estimated from historical appointment book-
ing data (e.g., Figure 1) considering an estimated response rate to reflect vaccine hesitancy. Given
that such booking pattern data was collected during the phase when vaccine supply and the VC
capacity were sufficient compared to the number of the eligible population in the subgroup, the
booking patterns were not censored and reflected the general booking behavior when residents have
been invited to the appointment system. Note that the throughput-maximizing demand arrival
pattern is not the actual demand arrivals. It is the hypothetical input for the booking limits opti-
mization to guarantee high throughput. Therefore, in the design of the booking limits, which is a
tactical operation decision, Dt is given and deterministic. It is also worth mentioning that under the
throughput-maximizing invitation schedule, the waiting time to get an appointment will be very
large. Therefore, we need to leverage the invitation schedule optimization (introduced in Section
3.2) to optimally stagger the demand arrival to further reduce the waiting time without sacrificing
throughput. In that optimization problem, we have to control the actual demand arrivals, which
are random. With the historical data on demand arrival patterns (Figure 1), we estimate partial
distribution information of the random daily arrival and adopt a distributionally robust model.
Let sut be the number of projected vaccine supplies that arrive on Day t. We treat sut to be the
supplies taking into account the supply uncertainty, such as worst-case supply quantities. Note
that such a treatment using the conservative estimate for supply sui is equivalent to adopting a
robust model. Let slt be the expired amount of vaccines at time t, which can be obtained given the
supply schedules and expiry date information. Ct denotes the vaccination capacity on Day t.
The key decision variables are the first and second-dose booking limits on Day t, denoted by
xt , t = 1, ..., T and yt , t = 1, ..., T ′ . The objective is to minimize the total cumulative number of
individuals waiting to get the first-dose appointments, denoted by Z(x, D). Let Z ∗ be the optimal
value, and we have the following formulation of the optimization problem.
Z ∗ = min Z(x, D)
x,y,z,v
min{t+k,T ′ }
X X X
(xi + yi + vi ) + yj ≤ sui , t = 1, ..., T ′ , (1d)
i:i≤t j=min{t+1,T ′ } i:i≤t
min{t+k,T ′ }
X X X
(xi + yi + vi ) + yj ≥ sli , t = 1, ..., T ′ , (1e)
i:i≤t j=min{t+1,T ′ } i:i≤t
′
xt + yt ≤ Ct , t = 1, ..., T , (1f)
xt , yt , zt , vt ≥ 0, t = 1, ..., T ′ . (1g)
where zt,t′ , t = 1, ..., T, t′ = 1, ..., T ′ denotes the number of individuals having the first dose at t and
the second dose at t′ , vt denotes the buffer for accounting the expired vaccines. The first three
constraints establish the two-dose interval requirement with a mini-stretch. Specifically, it ensures
that if the first-dose appointment is made on Day t, the second-dose appointment can only be made
on Day t + I to t + I + ms . The fourth constraint ensures that the vaccine supplies are sufficient
to fulfill the first and second booking limits and the next k days second-dose future bookings. The
fifth constraint models the expiration of the vaccines, and the last one is the capacity constraint.
The reformulation of Z ∗ requires a characterization of the objective function Z(x, D). To char-
acterize the waiting time, we adopt a reformulation technique in appointment scheduling literature.
For intraday appointment scheduling, patients arrive according to the appointment time, and the
waiting time captures the difference between the arrival time and the service starting time. Patients,
once they arrive, prefer to be served soonest. Therefore, to directly apply the reformulation tech-
nique, the following assumption is required.
Assumption 1. The individuals arrive in the appointment booking system on the day of their
most preferred appointment day.
However, this assumption generally does not hold in our context. Specifically, the “waiting time”
we consider is defined as the delay to get the first-dose appointment from the individual’s desired
appointment day to capture how the actual appointment deviates from the individual’s preferred
slot. For example, an individual enters the appointment booking system on Day 1 to book an
appointment two days later on Day 3; however, he can only find an available slot on Day 5. The
waiting time is (5 − 3) = 2. Assumption 1 implies an overestimated waiting time of (5 − 1) = 4.
In general, tracking the waiting time of each individual in such an appointment system is complex.
It may be driven by the desire of the individual to book an ideal appointment slot later into
the calendar, for instance, a preference to find a slot on a weekend vs. a weekday. Hence not
all the physical waiting times experienced in the appointment system are driven by scheduling
issues. Furthermore, the booking process of different individuals may crisscross in the system —an
individual who logs in to the appointment system later may actually get an appointment earlier
than those who enter the system earlier than him. To overcome this challenge, we can show that
Assumption 1 can be valid by a simple exchange of appointment slots for those who crisscross each
other if people prefer to have the closest appointment to their preferred day. Consider, for example,
invitee A arrives on Day 1 and prefers an appointment on Day 4; invitee B arrives on Day 2 and
prefers an appointment on Day 3. Currently, there is no slot available on Day 3 and only one slot
left on Day 4. Given that invitee A arrives early, he can take the one slot on Day 4. Invitee B can
get the earliest appointment on Day 5 and incur a waiting time of 2 days. With this assumption,
we interchange the arrival sequence of A and B, so that invitee B is assumed to arrive early on Day
3 and takes up the only one slot on Day 4 with a waiting time of one day. Invitee A is assumed to
arrive on Day 4 and fails to book his desired slot and get an appointment on Day 5 with a waiting
time of one day. Despite the deviations in tracking each individual’s waiting time, this assumption
does not affect the calculation of the total waiting time for all the individuals. Given that all the
individuals prefer to have the closest appointment to their preferred day, we can exchange any
two individuals in the system —the overestimation and underestimation offset each other. That is,
we can re-assign the appointment slots to different individuals to ensure that first-come-first-serve
holds in the appointment system, without affecting the aggregate waiting times of all in the system.
This simple transformation simplifies the problem significantly, but the price to pay is that we only
have aggregate waiting time information and could not piece together the individual’s waiting time
experienced in the system. With this, we can ensure the validity of Assumption 1.
In the rest of the paper, instead of directly working on the total waiting time for the first doses,
we consider minimizing the cumulative number of people waiting for the first doses. The average
waiting time can be obtained from the cumulative number of people waiting by factoring in the
throughput, using Little’s Law. Under Assumption 1, we can iteratively characterize the objective
function, Z(x, D), that is, the total cumulative number of people waiting for the first doses. Let
wt denote the number of people waiting for the first dose on Day t, t = 1, ..., T .
We assume once an invitation is issued, invited individuals have preference over the first-dose
appointment date represented by a choice probability, i.e., for an appointment on Day t, an indi-
vidual i will prefer it with probability pit . We model the heterogeneous choice probability among
different individuals by treating the choice probability as a random variable, denoted as p̃t . Given
that it is difficult to fully and accurately characterize the joint distribution of p̃t , t = 1, ..., T , we
assume only the first two moments are known, denoted as µ and Σ. Note that µ is proportional to
D in the previous section. We adopt a distributionally robust perspective and optimize the invi-
tation schedule to hedge against the worst-case scenario of distribution. Specifically, let P + (µ, Σ)
denote the uncertainty set of p̃t , defined as follows.
E[p̃] = µ, E[p̃p̃T ] = Σ,
P + (µ, Σ) = p̃ = (p̃t )Tt=1 P
T
p̃t = 1, a.s., p̃ ≥ 0, a.s..
t=1
T
x∗t + wT − rN , where wT is the number
P
Note that the total number of unfilled booking slots is
t=1
of individuals waiting for an appointment at time T (the end of the horizon). Therefore,
Z(n, x∗ ) can be similarly reformulated as a linear program using the iterative waiting time deduc-
tion. The following is the dual problem of the linear program. Due to strong duality, it is an
equivalent characterization of Z(n, x∗ ).
!
T ˆ
t/P
I+1 T
Z(n, x∗ ) = max rnm p̃t−(m−1)Iˆ − x∗t αt + λ( x∗t − rN )
P P
αt ,βt t=1 m=1 t=1
s.t. αt − αt−1 − βt = −(1 − λ), t = 2, .., T, (3)
−αT − βT +1 = −1,
αt , βt ≥ 0, t = 1, ..., T.
3.3.1. Copositive Program Reformulation We rewrite the inner linear problem (3) in a
general form as follows with a proper definition of coefficient matrices, A, B and C(n) which is
linear in the first-stage decision n.
T
Z(n, x∗ ) = max p̃T C(n)α − x∗ T α + λ( x∗t − rN )
P
αt ,βt t=1
α (4)
s.t. A B =b
β
αt , βt ≥ 0, t = 1, ..., T.
The inner distributionally robust problem, max E[Z(n, x∗ )] is therefore, a problem of find-
p̃∼P + (µ,Σ)
ing the moment bound of a linear optimization problem with random objective coefficients. To
solve this problem, the main challenge lies in the evaluation of the expectation under the worst-
case distribution. This requires us to obtain an explicit characterization of the ambiguity set P + .
There has been development in the distributional robust optimization field on reformulations using
completely and copositive programs (e.g., Natarajan et al. 2011, Xu and Burer 2018a,b). Following
the same line, we can show that inner distributionally robust problem, max E[Z(n, x∗ )] is also
p̃∼P + (µ,Σ)
equivalent to a completely positive program.
Consider the following completely positive program with decision variables, qα , qβ , Xα , Xβ , Yα , Yβ
and Xαβ . Proposition 1 establishes the equivalence between this completely positive program and
the inner distributionally robust problem.
T
max ⟨C(n), Yα ⟩ − x∗ T qα + λ( x∗t − rN )
P
t=1
qα
s.t. A B = b,
qβ
Xα Xαβ T
diag A B T AB = b ◦ b, (5)
Xαβ Xβ
T T T
1 µ qα qβ
µ Σ Yα Yβ
q Y T X X ≽cp 0.
α α α αβ
qβ YβT Xαβ
T
Xβ
By considering its dual copositive program, we have the following theorem that reformulates the
invitation schedule design problem.
4. Agent-Based Simulation
We develop an agent-based simulator to evaluate the performance of different rollout policy can-
didates that are operated under optimized decisions, that is, with the optimal booking limits and
invitation schedules. In the simulation, each VC is modeled with finite capacities, and every invitee
is modeled as an independent agent. The inputs of the simulator consider response rate, vaccine
preference, demand arrival patterns (based on Figure 1), VC preference, the trade-off between
location and waiting time for the appointment, etc. Other inputs for the simulation are derived
from the policy settings and the optimal booking limits and invitation schedules.
Each invitee’s preference of vaccine types i, vaccination center s, and appointment time t is
generated by a discrete choice model with utility,
where, for each invitee j, f j (i) ∈ {0, 1}, ∀i, represents a strict preference for one or more vaccine
types i. If an invitee j prefers Type k, f j (k) = 1 and f j (i) = 0 for all the i ̸= k. For the case
that the invitee j has no vaccine preference among vaccine types, f j (i) = 1 for all i. The location
preference is denoted as g j (s), which takes a value between 0 and 1. It can be determined by the
proximity of the vaccination site, the invitee demographic, etc. The time preference is denoted as
hj (t), which takes a value between 0 and 1. We derive the time preference from the arrival demand
patterns of early invited batches (Figure 1). ϵj (s, t) is a random noise term added to capture other
unobserved factors affecting the invitee’s preference for location and time, and we assume it follows
a normal distribution with mean 0 and standard deviation 0.05. At each time period, the invitees
are processed in a strict wave ordering to ensure that the earlier wave invitees have priority to
choose a slot over the later waves. If the model specifies a vaccine preference, it is handled by
setting f j (i) appropriately for the proportion of the invitees. The location choice is restricted to a
subset that administers the preferred vaccine type. In case of no preference, the invitee can freely
choose among all locations. Invitees will choose a slot (s, t) that gives the highest utility and move
to the second-highest if (s, t) is not available or if (s, t + I (i) ) to (s, t + I (i) + ms ) are not available
for the second dose. The availability of a slot is subject to the supply and capacity constraints, as
well as the booking limits set by the optimization output (except for the holdback policy). A slot
(s, t) can be chosen only if both the first and second slots can be booked.
We validate the simulation model using past vaccination data in Singapore. Specifically, we use
eight-week data starting from 29th March when Singapore implemented the holdback policy. Our
model is initiated with the inputs from 29th March and simulates the future first-dose take-ups if
the holdback policy is adopted. We use a simple log-normal distribution (Figure 3a) to calibrate
the expected booking pattern based on Figure 1. The sum squared error between the calibrated
curve and the average arrival pattern in Figure 1 is 0.69%. We observe the growth of first-dose
vaccinations under varying response rates. Comparing our prediction with the actual vaccination
data, we find that our model can accurately match the actual performance when the response rate
is 90% (Figure 3b). Note that the actual release of appointment slots is reviewed and updated
continuously by a team of planners in MOH, whereas our prediction is based on a single run of
the model. The degradation of the prediction accuracy reflects a shift to a more aggressive release
policy subsequently taken by the planners.
(a) Calibrated appointment booking (b) Prediction error of first dose vaccina-
arrival pattern. tions.
We have developed a comprehensive system called VaxOS, which serves as a general policy eval-
uation tool. This system takes various inputs into account, including (i) policy switching times,
safety lead time, stretch interval, vaccination center (VC) capacity, and more; (ii) the current
booking status in the system; (iii) the supply schedule of vaccines; and (iv) estimated values such
as arrival distribution, response rates, vaccine preferences, and other relevant factors. VaxOS gen-
erates several valuable outputs, including (i) the optimal booking limits for first and second-dose
appointments; (ii) an invitation schedule that optimizes the vaccination process; (iii) simulated
daily appointment bookings for both first and second doses; (iv) simulated waiting times experi-
enced by individuals; and (v) additional metrics such as expected daily reserve, inventory levels,
VC occupancy, and more. The outputs from VaxOS have been utilized by the Singapore MOH
Taskforce as policy benchmarks for validation and comparison. This highlights the significance of
VaxOS in providing valuable insights and aiding decision-making processes for policy evaluation
in the context of COVID-19 vaccination.
Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Type 1 (×1000 doses) 60 100 100 100 100 60 60 60 60 60 40 40 100 100
Type 2 (×1000 doses) 200 0 50 0 75 0 0 150 0 150 0 200 0 200
Table 2 Vaccine Supply Schedule
Given that the supply is the bottleneck, for the booking limits optimization problem, the total
number of people considered under the throughput-maximizing invitation schedule for each vaccine
type is obtained by the total regimens of each vaccine type supplied over the planning horizon
adjusted by an assumed take-up rate, so that the total expected vaccine supply matches with the
T T′
(i)
sut (i) /2, where r is
P P
number of people who show up, i.e., for each vaccine types i,, r Dt =
t=1 t=1
set to be 80%. For the invitation schedule design problem, given that the first-dose booking limits
serve as the upper bound of the total number of people that can take vaccines, the total number of
2 P
T
P (i)
people to invite is obtained from the total first-dose booking limits, i.e., rN = xt . Invitations
i=1 t
are initiated in a biweekly manner over 5 waves, i.e., Iˆ = 14 days, M = 5. The first moment µ of
the joint distribution of p̃ = (p̃t )Tt=1 is based on the log-normal distribution (Figure 3a).The second
moment can capture the variability in the arrival patterns for different subpopulation groups. The
covariant matrix is assumed to be a diagonal matrix with the (i, i)-th entry equals µi (1 − µi ),
motivated by a Bernoulli distribution. It can be shown the uncertainty set, P + (µ, Σ), is not empty
(i)
with these first two moments. The throughput-maximization pattern, Dt , is proportional to µt ,
T T
(i)
and is obtained as (r Dt ) × µt (= ( sut (i) /2) × µt ), for i = 1, 2.
P P
t=1 t=1
We compare the optimal rollout policies under different levels of second-dose reserve specified
by parameter k, with the holdback policy. We first consider the policy that is adherent to two-dose
intervals with a mini-stretch of 4 days. In this case study, the manufacturer recommended two-dose
intervals for Type 1 and Type 2 vaccines are I (1) = 21 and I (2) = 28, respectively. We also conduct
analysis for stretch policies with I (1) = I (2) = 42 days and 49 days for both types of vaccines. Details
of stretch policy analysis are given in E-Companion EC.3.
In particular, Figure 4a shows the optimal cumulative total (first dose and second dose) booking
limits for Type 1 vaccines at various levels of the second-dose reserve, from the most aggressive
policy with no reserve (k = 0) to a conservative policy with a full two-dose interval reserve (k =
21). The most aggressive policy touches the cumulative supply curve and achieves the highest
throughput. It is also observed that a more regressive policy that holds fewer reserves for the second
doses can help reduce the first-dose waiting time.
Figure 4b shows the optimal cumulative total booking limits for Type 2 vaccines. Notice that all
the cumulative bookings limit curves do not touch the cumulative supply curve after Day 49 given
the limited vaccination capacity. Interestingly, under this numerical case, the most aggressive policy
that keeps no reserve performs worst in terms of throughput. Contrary to conventional belief, it also
gives rise to a longer average waiting time than, for example, the one-week and two-week reserve
policies. To understand this observation, we plot the daily first and second-dose booking limits of
the no-reserve policy and the policy with a two-week reserve (k = 14) in Figure 5. In Figure 5a,
(a) Type 2 Vaccine No Reserve Policy k = 0 (b) Type 2 Vaccine 2-week Reserve Policy k = 14
no reserve policy aggressively administers the first dose from Day 1 to Day 28. The corresponding
second-dose intakes start from Day 29 given a 28-day two-dose interval. The need to administer
second doses blocks the first-dose bookings during those days under limited vaccination capacity.
The large demand for second doses also exhausts all the supply and no bookings are possible from
Day 43 to Day 49 (also can be seen in Figure 4b, there is a flat region of total cumulative bookings
from Day 43 to Day 49). This is also why the first wave of first-dose bookings ends on Day 10
because no vaccines are available for the corresponding second doses. The blocking phenomenon
leads to fewer first-dose booking limits in the rest of the horizon and hurts the overall throughput.
Interestingly, by keeping the right amount of second-dose reserve, the blocking is alleviated. For
example, in Figure 5b, firstly, we observe that the first round of first-dose bookings goes beyond
Day 10. When the total supply exhausts, we can allow the second-dose bookings using the reserve.
It would lift the constraint on the first-dose bookings from Day 10 onward. Secondly, because
of the requirement of keeping a second-dose reserve, the first-dose bookings cannot be set too
aggressively in earlier weeks. The corresponding second-dose bookings would not block the future
first doses that significantly. As highlighted in Figure 5, for the no-reserve policy, between Day
49 to Day 70, only limited first-dose bookings are allowed in order to accommodate the required
second doses. Whereas for the two-week reserve policy, we observe significantly more first-dose
bookings. Therefore, early completion of the first-dose administration is possible.
The counter-intuitive policy performance is observed for Type 2 vaccine but not for Type 1
vaccine. This is because Type 2 vaccine has a more stringent vaccination capacity and more irregular
(longer delay) supplies. In this case, aggressively administering the first doses, though, benefits the
throughput in the early days, may result in blocking that hurt the future throughput. When the
blocking is sufficiently prominent, the overall throughput might be sacrificed.
The cumulative booking curves such as those shown in Figure 4 can provide a guideline to
determine a proper level of second-dose reserve. For Type 1 vaccine where there is no significant
blocking, we can choose a k that gives a reasonable amount of reserve to hedge against the supply
risk. For Type 2 vaccine, a good choice is the two-week reserve policy where throughput is highest,
waiting time is lowest, and the reserve amount is sufficient to hedge against supply disruption. The
two-week reserve booking limits for both two vaccine types can be the recommended policies.
T
P
the total cumulative number of people waiting for the first doses, wt , has a significantly larger
t=1
magnitude than the unfilled bookings, wT , a proper λ should be scaled to a high value close to 1.
In our study, we also find that the optimal invitation schedule is less sensitive when λ is far below
1. The trade-off between the waiting time and throughput becomes more salient in influencing the
optimal invitation schedules when λ ≥ 0.95. In the rest of the policy comparison, we use λ = 99.99%
to demonstrate the “best-throughput worst-waiting-time” case. For the case that the population
exhibits a prevalent vaccine preference, we study the scenario of l1 = 0.4, that is 40% of them
prefer Type 1 over Type 2. Table 4 shows the invitation schedules for the one-week risk and the
recommended two-week reserve policies compared with the holdback policy.
No Vaccine Preference (λ = 99.99%) With Vaccine Preference (λ = 99.99%, l1 = 0.4)
Recommended Recommended
Wave Holdback 1-Week Risk 1-Week Risk
Policy Policy
1 225000 355524 377122 376477 390724
2 156000 130106 120147 185803 183442
3 147000 74171 72243 87790 84387
4 169000 90759 88176 235431 247931
5 169000 251315 269420 16374 20624
Table 4 Optimal Invitation Schedules
The holdback policy invites people purely based on the supply schedule. Therefore, it does not
respond to the vaccine preference information. One-week risk and recommended two-week reserve
optimal invitation schedules are more aggressive for both cases with and without vaccine preference.
When there is no vaccine preference, the optimal invitation schedule aggressively invites people in
the first two waves to achieve higher throughput. Later waves are mild to control waiting time,
and the last wave catches up. When there is a vaccine preference for Type 2 over Type 1, more
people would be waiting for the first dose of Type 2 vaccines, and more unfilled booking limits for
Type 1 vaccines. The “best-throughput worst-waiting-time” optimal invitation schedules are more
aggressive so that unused Type 1 booking limits can be filled up.
Compared to the holdback policy, the one-week risk policy that holds fewer second doses as
reserve improves the throughput at a minimal cost of waiting time. The recommended two-week
reserve policy performs the best in both dimensions even under the “best-throughput worst-waiting
time” case (λ = 99.99%), achieving the highest throughput without increasing the waiting time, as
it is designed with the second-dose reserve level that minimizes the blocking.
When a vaccine preference is not matched with the vaccine supply or capacity, both throughput
and waiting time might become worse. In our numerical study, Type 1 vaccine has a higher supply
and vaccination capacity but is not preferred compared with Type 2 vaccine (with l1 = 0.4). In Table
5, we observe a significantly lower throughput and higher waiting time for all policies compared
to their corresponding ones in the no-vaccine preference case. The throughput of Type 1 vaccines
drops by 25 − 30% due to less demand. As more people prefer Type 2 vaccine, the average waiting
time increases from 0 to 9 to 12 days.
Compared with the holdback policy, the one-week risk, and the recommended policies can effec-
tively use the vaccine preference information to improve the throughput without significantly
increasing the average waiting time (from 5 days to 7 days). More interestingly, the recommended
two-week reserve policy improves upon the one-week reserve policy by choosing the right amount
of second-dose reserve. The blocking phenomenon is more severe for the case with vaccine prefer-
ence where Type 2 vaccine supply and capacity become tighter limiting the vaccination progress.
Keeping the right amount of second-dose reserve can help alleviate the blocking and thus achieves
better performance in both dimensions.
utility model which specifies a vaccine preference, a random-ordered subset for VC preferences,
and preferences for the appointment time. The Taskforce carefully designed the VC distribution
in Singapore to be near population centers and it is reasonable to assume that an individual will
prefer to get an appointment at a nearby VC’s at the earliest time slot. Due to the sensitive nature
of the supply and capacity data, we have scaled and perturbed the values while performing the
optimization and simulation and masked the y-axis in the charts.
In the following, we focus on presenting the policy evaluation study we conducted to address
the real challenges faced by Singapore Taskforce when witnessing an increase in COVID-19 cases
in early 2021. Specifically, we aim to answer the following questions. First, does implementing an
aggressive policy always help improve the throughput? Secondly, when is a good time to switch
to an aggressive policy in Singapore? Thirdly, how long should the stretch interval be and how
many second-dose reserves should keep? Note that the numerical results in this study are obtained
from the simpler linear programming-based models for both booking limits and invitation schedule
optimization, which is the framework used by the Taskforce.
For the holdback policy, the booking limits and invitations are computed using a heuristic as
used in actual operations. The booking limit is set to exactly half of the supply available at
the time after discounting for existing appointments and the invitation schedule is computed by
scaling the booking limit by the response rate. For all other policy candidates, if not given specific
names, we label them based on the chosen two-dose stretch and the second-dose reserve level.
We consider the planning horizon starting from March 29th, 2021, when the holdback policy was
implemented. There were projected 8 million vaccines available for the next 20 weeks. We simulate
various policy candidates and benchmark them with adopting the holdback policy throughout the
planning horizon. Given that the main objective for the policymaker is to accelerate the first-dose
intakes facing the challenges of increasing COVID-19 cases, in our numerical studies, we focus on
the comparison of the first-dose vaccination throughput under different policies. The performance
comparisons in terms of two-dose vaccination rates are relegated to E-Companion EC.4.1.
Aggressive Policy stretches the two-dose interval to 6 weeks for the Pfizer vaccine and 8 weeks for the
Moderna vaccine, which doubles the two-dose intervals in holdback policy. It will still nevertheless
maintain enough reserve of the vaccines for the next 3 weeks of second dose appointments. Invitation
schedules were sent out to the invitees at two-week intervals.
Figure 6 Comparison of booking limits between the holdback and the Aggressive Policy. The black curve is the
cumulative vaccine supplies. The red (purple) curve is the cumulative total number of first and second
dose booking limits under the Aggressive Policy (holdback policy), with the dashed curve and dotted
curve corresponding to the first and second dose booking limits, respectively.
Figure 6 shows the optimized cumulative first, second, and total booking limits for these two
policies. Comparing the two policies based on the cumulative total and the first booking limits
curves (solid and dash lines), Aggressive Policy does not always dominate the holdback policy
throughout the planning horizon. In particular, till the end of Week 10, the total number of
appointment slots released under the Aggressive Policy is comparable with the holdback policy.
This is due to the limited supply in Singapore during Week 6 to Week 8 that capped the growth of
the booking curves. A higher growth rate becomes visible only after Week 12. However, the second
dose administration is delayed due to the long stretch under the Aggressive Policy.
Fortnightly Invitation Schedule (in thousands) Cumulative Values
Period Holdback Aggressive Hybrid Holdback Aggressive Hybrid
Supply Supply
(Week) (A) (B) (C) (A) (B) (C)
0 1,160 470 1,804 470 1,160 470 1,804 470
2 452 282 0 282 1,613 752 1,804 752
4 450 460 0 460 2,063 1,212 1,804 1,212
6 632 300 0 300 2,695 1,513 1,804 1,513
8 481 136 50 1,214 3,175 1,648 1,854 2,727
10 455 448 831 0 3,631 2,097 2,685 2,727
12 455 121 396 654 4,086 2,217 3,081 3,381
14 655 698 1,114 998 4,741 2,915 4,195 4,378
16 1117 698 908 716 5,858 3,614 5,104 5,094
18 1117 698 - 6,975 4,312 - -
20 1117 698 - 8,092 5,010 - -
Table 6 ”Holdback Policy (A)” implements the recommended stretch with full reserve, ”Aggressive Policy
(B)” implements a 6-8 Week stretch with a 3-week reserve, and ”Hybrid Policy (C)” implements the Holdback
Policy for the first 8 weeks followed by the Aggressive Policy.
The downside of implementing such an aggressive policy can be illustrated by the corresponding
optimal invitation schedule shown in Table 6. The table shows fortnightly invitation schedules
for different policies until 5 million invitations are reached. With the assumed 80% response rate,
an estimated 4 million people will be vaccinated which is the maximum with the 8 million doses
available in the planning horizon. Under an Aggressive Policy (B), a large number are invited in
the first wave. This, together with the slow supply, leads to a long pause afterward, and the next
group is invited only after week 8. This is in contrast with the holdback policy that has more
steady and regular invitations. The comparison shows that an aggressive policy implemented too
early when supply cannot catch up may not realize the benefits as expected. The long pause of
the first dose appointment due to supply shortage in Aggressive Policy is undesirable: It leaves
an impression of vaccine inaccessibility and would result in serious public implications over the
vaccination campaign. The large initial invitation wave also results in a longer lead time for an
individual to book an appointment and negatively impacts their booking experience.
To show the robustness of the insights, we simulated the first-dose intakes of other stretch
policies, such as 4(5)-week stretch for Pfizer (Moderna) with 3-week reserve, and 5(7)-week stretch
for Pfizer (Moderna) with 3-week reserve. Though they are less aggressive with shorter stretch
intervals, the growth patterns are similar if the corresponding policies had been implemented at
the beginning of the planning horizon. Details are given in E-Companion EC.4.2.
Figure 7 (Color online.) Comparison of dynamic policies with different switching times. The purple curve is the
cumulative number of first doses vaccinated under the holdback policy. The green curve corresponds
to the Hybrid Policy (i.e., switching from holdback to Aggressive Policy in Week 8). The green dashed
curve corresponds to the dynamic policy that switches from holdback to Aggressive Policy in Week 4.
Figure 8 (Color online.) Performance of different policies to reach 4 million total 1st Dose vaccinations. At 0W,
1 million people have been vaccinated for the first dose. The numbers on the figure are additional first
doses administered.
The intrinsic value of the policy choice becomes apparent when considering its impact on mitigat-
ing public health risks and stabilizing the economy. Throughout the COVID-19 pandemic, although
it is challenging to quantify directly, higher vaccination rates have demonstrated a reduction in
strain on healthcare systems, prompting governments worldwide to gradually ease restrictions as
a significant portion of the population becomes vaccinated. In this context, the time required to
vaccinate a substantial proportion of the population directly correlates with both the public health
risk and economic consequences. Our simulation results indicate that the implementation of the
Hybrid Policy saves the nation approximately 23 days in vaccinating a total of 3 million residents
with at least one dose, and nearly 30 days in vaccinating up to 4 million residents with at least
one dose, in comparison to the holdback policy (see Figure 8). This significant time reduction is
a testament to the efficacy of the Hybrid Policy in accelerating the first-dose vaccination rate.
Our optimization + simulation framework recommends adopting the Hybrid Policy that switches
from holdback to a policy that doubles the two-dose interval and keeps 3 weeks reserve for the
second-dose appointments in mid or later May (8 weeks after March 29). In fact, on May 19, MOH
announced a modification to its vaccination program - “those who register for COVID-19 vaccina-
tion will have their second dose scheduled six to eight weeks after the first, instead of three to four
weeks later.” This tweak is to “protect as many people as possible, and as soon as possible” (Lai
2023). This switch timing and stretch interval matched exactly with the recommended policy.
To further assess the impact of our contribution and verify the correspondence between our
predicted performance of the Hybrid policy and the actual vaccination outcome after implementing
the stretch policy, we conducted a comparison between our simulated first-dose intakes and the
public records of the first-dose vaccination data (Figure 9). Following the implementation of the
6-8 weeks stretch policy in Singapore, the number of first doses administered increased to 3 million
doses, exhibiting an average growth rate of 1.5. Subsequently, it further rose to 4 million doses
with an average growth rate of 2.7. Notably, our simulated growth rates, based on the available
information as of March 29, closely align with the actual growth rates, with values of 1.2 and 2.1
before and after the policy switch, respectively. In fact, this strategic change enabled the nation
to safeguard up to 75% of its population across all age groups by September 2021, when the first
major wave of the Delta variant hit the nation.
Figure 9 Comparing policy performance against the actual vaccination progress in Singapore.
The above analysis relies on calibrated demand arrival patterns using a log-normal distribution
and an 80% response rate. A robustness study on these inputs is given in E-Companion EC.4.3.
7. Concluding Remarks
The policy-makers have been adaptively revising the vaccine rollout policies based on COVID-
19 cases as well as vaccine availability and supply chain considerations. Effective scheduling of
COVID-19 vaccinations is crucial in reducing both the spread of the virus and the severity of its
outcomes. The optimization and simulation-based framework is critical in supporting these policy
changes, by facilitating comparisons among potential policy candidates, quantifying the impact of
each policy, and eventually providing policy recommendations. Before mid-May, Singapore adopted
the non-stretch policy that is similar to a holdback policy. Our framework shows that switching to
a stretch policy at the right timing and with a good choice of the stretch interval can significantly
accelerate the vaccination rate. In particular, the recommended hybrid policy has been shown to
save up to 30 days to reach 4 million first-dose vaccinations. In addition, the optimization framework
also provides operational guidance on how to design appointment booking systems and invitation
schedules to realize the benefits. Singapore was able to vaccinate most of its adult population before
the Delta wave hit in September 2021 with the successful switch from the holdback to stretch
policy. The 30 days’ savings paved the way to rapidly provide elderly 60 years old and above with
a booster that was critical in mitigating the impact of the Delta wave.
While the COVID-19 pandemic is waning, it has still had a significant impact on global health
and the economy. The developed open-source web service, VaxOS, integrates appointment booking
limits, invitation schedule design solvers, and a flexible simulation model. It is fully parameterized
to accommodate a range of scenarios and can be accessed through a user-friendly web-based dash-
board. The Singapore vaccination team has already implemented VaxOS to manage its appointment
booking process. We believe that these tools, which we have made available via links (withheld
during the paper review), can be instrumental in policy design and operational decision-making in
other countries and regions.
The research conducted on COVID-19 provides valuable insights for handling future pandemics
and outbreaks. These optimization tools have far-reaching applications beyond COVID-19, as they
can be invaluable in responding to any pandemic that requires rapid vaccination of population
groups while facing uncertainties in demand and supply. As most vaccines have a waning effect
and many have already followed multiple-dose regimens (WHO 2022), it is likely that the multi-
dose regimen might be followed in a future pandemic and as part of “preparedness”. In addition
to investments in vaccine development and manufacturing, it is important to ensure operational
preparedness is also prioritized. We hope that our work contributes to the knowledge base for
future endeavors.
Acknowledgments
The authors gratefully acknowledge the suggestions and support from the Ministry of Health, Singapore,
vaccine operations team in this collaboration.
References
Arifoğlu K, Deo S, Iravani SM (2012) Consumption externality and yield uncertainty in the influenza vaccine
supply chain: Interventions in demand and supply sides. Management Science 58(6):1072–1091.
Arifoğlu K, Tang CS (2021) A two-sided incentive program for coordinating the influenza vaccine supply
chain. Manufacturing & Service Operations Management .
Bai M, Chen G, Li C (2023) First dose or second dose? a study of vaccine policy with supply and capacity
constraints .
Bennouna A, Joseph J, Nze-Ndong D, Perakis G, Singhvi D, Lami OS, Spantidakis Y, Thayaparan L, Tsiour-
vas A (2022) Covid-19: Prediction, prevalence, and the operations of vaccine allocation. Manufacturing
& Service Operations Management .
Booth W, Adam K (2021) Can I choose my COVID vaccine? strong opinions on Oxford vs. Pfizer emerge
in U.K. .
Calafiore GC, Parino F, Zino L, Rizzo A (2023) Dynamic planning of a two-dose vaccination campaign with
uncertain supplies. European journal of operational research 304(3):1269–1278.
Cayirli T, Veral E (2003) Outpatient scheduling in health care: a review of literature. Production and oper-
ations management 12(4):519–549.
Chen X, Li M, Simchi-Levi D, Zhao T (2020) Allocation of covid-19 vaccines under limited supply. MedRxiv
2020–08.
Chick SE, Mamani H, Simchi-Levi D (2008) Supply chain coordination and influenza vaccination. Operations
Research 56(6):1493–1506.
Dai T, Song JS (2021) Transforming covid-19 vaccines into vaccination: Challenges and opportunities for
management scientists. Health care management science 24(3):455–459.
de Klerk E, Pasechnik DV (2002) Approximation of the stability number of a graph via copositive program-
ming. SIAM Journal on Optimization 12(4):875–892.
Duijzer LE, van Jaarsveld WL, Wallinga J, Dekker R (2018) Dose-optimal vaccine allocation over multiple
populations. Production and Operations Management 27(1):143–159.
Engineer FG, Keskinocak P, Pickering LK (2009) OR practice—Catch-up scheduling for childhood vaccina-
tion. Operations Research 57(6):1307–1319.
GovUK (2020) Statement from the UK chief medical officers on the prioritisation of first doses of COVID-19
vaccines .
Green LV, Savin S (2008) Reducing delays for medical appointments: A queueing approach. Operations
Research 56(6):1526–1538.
Gupta D, Denton B (2008) Appointment scheduling in health care: Challenges and opportunities. IIE trans-
actions 40(9):800–819.
Hall VG, Ferreira VH, Wood H, Ierullo M, Majchrzak-Kita B, Manguiat K, Robinson A, Kulasingam V,
Humar A, Kumar D (2022) Delayed-interval bnt162b2 mrna covid-19 vaccination enhances humoral
immunity and induces robust t cell responses. Nature Immunology 23(3):380–385.
Karakaya S, Balcik B (2023) Developing a national pandemic vaccination calendar under supply uncertainty.
Available at SSRN 4380147 .
Lai L (2023) S’pore delays 2nd Covid-19 vaccine dose to 6-8 weeks later; those aged 40-44 can regis-
ter for jabs from Wednesday — straitstimes.com. https://www.straitstimes.com/singapore/
spore-delays-2nd-vaccine-dose-to-6-8-weeks-later-those-aged-40-44-can-register-for-jabs,
[Accessed 21-Apr-2023].
Mak HY, Dai T, Tang CS (2022) Managing two-dose covid-19 vaccine rollouts with limited supply: Oper-
ations strategies for distributing time-sensitive resources. Production and Operations Management
31(12):4424–4442.
Martinez DR, Ooi EE (2022) A potential silver lining of delaying the second dose. Nature Immunology
23(3):349–351.
Moghadas SM, Vilches TN, Zhang K, Nourbakhsh S, Sah P, Fitzpatrick MC, Galvani AP (2021) Evaluation
of covid-19 vaccination strategies with a delayed second dose. PLoS biology 19(4):e3001211.
MOH (2020) Ministry of Health Singapore, Expert committee submits recommendations on Singapore’s
COVID-19 vaccination strategy.
Murray S (2021) Exclusive: Biden will release nearly all available vaccine doses in break from Trump admin-
istration policy of holding back stock for second dose. CNN politices .
Natarajan K, Teo CP, Zheng Z (2011) Mixed 0-1 linear programs under objective uncertainty: A completely
positive representation. Operations research 59(3):713–728.
Oceguera-Castillo VJ, Vital-Soto A, Olivares-Aguila J (2022) The use of scheduling in the covid-19 pandemic:
A systematic literature review .
OurWorldInData (2023) Total number of people who received at least one dose of COVID-19 vaccine
— https. https://ourworldindata.org/grapher/people-vaccinated-covid?time=2021-03-29.
.2021-07-31\&country=~SGP, [Accessed 21-Sept-2021].
Paltiel AD, Schwartz JL, Zheng A, Walensky RP (2021) Clinical outcomes of a covid-19 vaccine: Imple-
mentation over efficacy: Study examines how definitions and thresholds of vaccine efficacy, coupled
with different levels of implementation effectiveness and background epidemic severity, translate into
outcomes. Health Affairs 40(1):42–52.
Parrilo PA (2000) Structured semidefinite programs and semialgebraic geometry methods in robustness and
optimization. Ph.D. thesis, California Institute of Technology.
Payne RP, Longet S, Austin JA, Skelly DT, Dejnirattisai W, Adele S, Meardon N, Faustini S, Al-Taei S,
Moore SC, et al. (2021) Immunogenicity of standard and extended dosing intervals of bnt162b2 mrna
vaccine. Cell 184(23):5699–5714.
Robertson JF, Sewell HF (2021) Covid-19 vaccine efficacy data: solid enough to delay second dose? The
Lancet 397(10291):2248–2249.
Robertson JF, Sewell HF, Stewart M (2021) Delayed second dose of the bnt162b2 vaccine: innovation or
misguided conjecture? The Lancet 397(10277):879–880.
Silva PJ, Sagastizábal C, Nonato LG, Struchiner CJ, Pereira T (2021) Optimized delay of the second covid-19
vaccine dose reduces icu admissions. Proceedings of the National Academy of Sciences 118(35).
Talluri KT, Van Ryzin G (2004) The theory and practice of revenue management, volume 1 (Springer).
Tuite AR, Zhu L, Fisman DN, Salomon JA (2021) Alternative dose allocation strategies to increase benefits
from constrained covid-19 vaccine supply. Annals of internal medicine 174(4):570–572.
Voysey M, Clemens SAC, Madhi SA, Weckx LY, Folegatti PM, Aley PK, Angus B, Baillie VL, Barnabas
SL, Bhorat QE, et al. (2021) Single-dose administration and the influence of the timing of the booster
dose on immunogenicity and efficacy of chadox1 ncov-19 (azd1222) vaccine: a pooled analysis of four
randomised trials. The Lancet 397(10277):881–891.
Wang X, Jiang R, Qi M (2023) A drone-based robust location and scheduling problem with uncertain supply
for pandemic vaccine distribution. Available at SSRN .
WHO (2022) Immunization coverage — who.int. https://www.who.int/en/news-room/fact-sheets/
detail/immunization-coverage, [Accessed 07-May-2023].
Xu G, Burer S (2018a) A copositive approach for two-stage adjustable robust optimization with uncertain
right-hand sides. Computational Optimization and Applications 70:33–59.
Xu G, Burer S (2018b) A data-driven distributionally robust bound on the expected optimal value of uncer-
tain mixed 0-1 linear programming. Computational Management Science 15:111–134.
Yamin D, Gavious A (2013) Incentives’ effect in influenza vaccination policy. Management Science
59(12):2667–2686.
Yu S, Kulkarni VG, Deshpande V (2020) Appointment scheduling for a health care facility with series
patients. Production and Operations Management 29(2):388–409.
Zhang AZ, Bai M, Cui Y, Kong G (2021) No panic in pandemic: The impact of individual choice on public
health policy and vaccine priority. Available at SSRN 3763514 .
Supplementary Materials
EC.1. Proofs of Statements
Proof of Proposition 1. Firstly, by the construction of Problem (5), it is a relaxation of
max E[Z(n, x∗ )]. Let α∗ (p̃) and β ∗ (p̃) denote the optimal solution of the linear program (3)
p̃∼P + (µ,Σ)
given a realization of p. Define
qα := E[α∗ (p̃)], qβ := E[β ∗ (p̃)]
Xα := E[α (p̃)α (p̃) ], Xβ := E[β ∗ (p̃)β ∗ (p̃)T ]
∗ ∗ T
Given that the set of all completely positive matrices is a convex set, taking expectation preserves
the complete positiveness. Hence,
1 µT T
qα qβT
µ Σ Yα Yβ
≽ 0.
q
α YαT Xα Xαβ cp
qβ YβT T
Xαβ Xβ
We have Problem (5) as the relaxation of max E[Z(n, x∗ )] and the objective value of Problem
p̃∼P + (µ,Σ)
(5), Zcp ≥ max E[Z(n, x∗ )].
p̃∼P + (µ,Σ)
To see the equivalence, let qα ∗ , qβ ∗ , Xα∗ , Xβ∗ , Yα∗ , Yβ∗ and Yαβ
∗
, be the optimal solution to Prob-
lem (5), and consider the rank-1 decomposition of the completely positive matrix at this optimal
solution, i.e.,
T
1 µT T
qα qβT θk θk
µ Σ Yα Yβ p̂αk p̂αk
X
=
q
α YαT Xα Xαβ δk δk
k∈κ β
qβ YβT T
Xαβ Xβ δk δkβ
the feasible solutions to Problem (4) and δkα = δkβ = 0, ∀k ∈ κ0 . Similarly, given that µ and Σ are
feasible
first-two
moments such that the uncertainty set P + is nonempty (i.e., 1′ µ = 1, 1′ Σ1 =
T
1 µ PT p̂
1, ≽cp 0), we have t=1 θkt = 1, p̂θk ≥ 0, ∀k ∈ κ+ , and p̂k = 0, ∀k ∈ κ0 . We can rewrite the
µ Σ k k
decomposition as
T
T T T 1 1
1 µ qα qβ
p̂k p̂k
µ Σ Yα Yβ X 2 θk θk
α α
q Y T X X = θk δk
δk
α α α αβ θ
k∈κ k θk
T
qβ YβT Xαβ + β β
Xβ δk δk
θk θk
Assume a random vector p̃ and its corresponding feasible solutions (α∗ (p̃∗ ), β ∗ (p̃∗ )) follow the
∗
It can be easily verified this is a valid and feasible distribution that satisfies the first two moments
constraints of p̃. By the same argument as in Natarajan et al. (2011), Problem (5) is equivalent to
max E[Z(n, x∗ )]. Q.E.D.
p̃∼P + (µ,Σ)
Proof of Theorem 1. Consider the dual problem of the completely positive program (5), a copos-
itive program as follows.
T
min ρ + ν T µ + ⟨Θ, Σ⟩ + ϕT b + γ T (b ◦ b) + λ( x∗t − rN )
P
ν,Θ,ϕ,γ t=1
ρ ν T /2
ϕT A B /2
0 0T −x∗ T /2 0
(EC.1)
− 0∗ O C(n)/2 O
s.t. ν/2 Θ O ≽co 0
T T −x /2 C(n)/2 O O
A B ϕ/2 O A B diag(γ) A B 0 O O O
We have the invitation schedule optimization problem reformulated the copositive program.
Q.E.D.
Proof of Theorem 2. The proof follows the exact logic as Proposition 3 followed by Theorem 2.
We omit the details here.
min Z(n, x∗ )
M
P
s.t. nm = N (EC.2)
m=1
nm ≥ 0, m = 1, ..., M
T
x∗t + wT − rN , where wT is the number
P
Note that the total number of unfilled booking slots is
t=1
of individuals waiting for an appointment at time T (the end of the horizon). Therefore,
Recall that the average demand arrival pattern is represented by the proportion of invited people
choosing first-dose appointments on Day t, µt ; and r is the response rate. We, therefore, derive the
following linear program reformulation.
−1
TP T
x∗t − rN )
P
min (1 − λ) wt + wT + λ(
t=1 t=1
s.t. w1 ≥ rn1 µ1 − x∗1 ,
ˆ
⌊t/P
I⌋+1
wt − wt−1 ≥ rnm µt−(m−1)Iˆ − x∗t , t = 2, ..., T, (EC.3)
m=1
M
P
nm = N,
m=1
nm ≥ 0, m = 1, ..., M,
wt ≥ 0, t = 1, ..., T.
Note that the first two constraints are specifically derived from the iterative relationship among
waiting times, wt .
When there is vaccine preference, considering multiple types of vaccines and population prefer-
ences over different types, we modify the optimization model introduced above as follows. Denote
the proportion of the targeted subpopulation that prefers type i over the others as li , ∀i = 1, ..., B
B
P
such that li = 1. The invitation schedule design problem can be similarly reformulated as follows.
i=1
B
−1
TP T
i i i∗
P P
min (1 − λ) wt + wT + λ( xt − rN li )
i=1 t=1 t=1
s.t. w1i ≥ rn1 li µ1 − x∗1 i , i = 1, ..., B,
ˆ
⌊t/P
I⌋+1
wti − wt−1
i
≥ rnm li µt−(m−1)Iˆ − x∗t i , t = 2, ..., T, i = 1, ..., B. (EC.4)
m=1
M
P
nm = N,
m=1
nm ≥ 0, m = 1, ..., M,
wti ≥ 0, t = 1, ..., T, i = 1, ..., B.
Similar to response rate r, demand arrival pattern pt , t = 1, ..., T , the vaccine preference parameter,
li , i = 1, ..., B, can be estimated from past vaccination data, and adaptively updated as new data
becomes available.
Table EC.1 Average waiting time under throughput-maximizing invitation schedule (Stretch policy)
Six-Week Stretch Policy Seven-Week Stretch Policy
One-Week Two-Week Three-Week Four-Week One-Week Two-Week Three-Week Four-Week
No-Reserve No-Reserve
Reserve Reserve Reserve Reserve Reserve Reserve Reserve Reserve
Type 1 Vaccine 5.2 9.2 13.2 14.8 17.9 1.5 5.2 9.2 13.2 14.5
Type 2 Vaccine 30.7 30.3 39.5 37.2 41.4 22.0 26.7 34.3 34.2 39.4
The optimal cumulative total booking limits of Type 1 and Type 2 vaccines under six-week
and seven-week stretch policies are presented in Figure EC.1. The corresponding waiting times are
given in Table EC.1. Similar to the non-stretch policy, the blocking is more severe for the Type 2
than for the Type 1 vaccine. But compared with the non-stretch policy, a longer stretch would help
alleviate the blocking, and as seen in Figure EC.1, there is little blocking under the seven-week
stretch policy. The waiting time is significantly reduced compared with the non-stretch policy, and
it is shortest under the seven-week stretch policy. From the cumulative booking limits curve, we
can choose the value of k for each vaccine type based on the degree of blocking observed. Under
the six-week stretch policy, a good policy candidate would be a three-week reserve for both two
types of vaccines, which has a sufficient second-dose reserve amount and relatively mild blocking.
For the seven-week stretch policy, the policy with a four-week reserve for the Type 1 vaccine and
a three-week reserve for the Type 2 vaccine is recommended.
(a) Type 1 Vaccine (Six-Week Stretch) (b) Type 2 Vaccine (Six-Week Stretch)
(c) Type 1 Vaccine (Seven-Week Stretch) (d) Type 2 Vaccine (Seven-Week Stretch)
Figure EC.1 The Optimal Cumulative Total Booking Limits of the Stretch Policies
The optimal invitation schedules with and without vaccine preference are shown in Table EC.2.
We also generate the four-week reserve invitation schedule as a comparison with the recommended
policies. Compared with the optimal invitation schedules that are adherent to three/four-week dose
interval, the optimal stretch policies are much more aggressive due to longer dose-interval that can
accommodate more first-dose bookings.
The simulated performance of each policy is shown in Table EC.3 and Table EC.4. Note that the
stretch policies significantly improve the throughput compared with the non-stretch policies and
holdback policy, with only a slight increase in the waiting time under the “best-throughput worst-
waiting time” case (λ = 99.99%). The recommended six-week stretch policy and the recommended
seven-week stretch policy achieve the best performance.
Figure EC.2 Comparing the total number of doses (first and second) under different policies applied since week
0.
Figure EC.3 Comparing the total number of doses (first and second) for the Aggressive Policy (6(8) week stretch
for Pfizer(Moderna) with 3 Week reserve) applied at different starting dates.
Specifically, in Figure EC.2, during weeks 3-5, we can observe that the second dose reserve forces
the slowing down of the total vaccines administered. It is easy to notice that any further acceler-
ation of the first dose will continue to dip into the reserve inventory, which explains the blocking
phenomenon experienced under aggressive policies with limited supply conditions. A similar phe-
nomenon can be observed in Figure EC.3 when the 6(8)-week stretch for Pfizer(Moderna) with
3-week reserve policy is applied starting from week 4 (notice the slowing of total doses administered
between week 8 and week 11).
In Figure EC.4, we compare the various dynamic policies when the Aggressive Policy is applied
at week 8. It is interesting to notice that the consequence of the total vaccine administered remains
pretty similar under varying modes of aggression, as all of these scenarios are eventually constrained
Figure EC.4 Comparing the total number of doses (first and second) under different aggressive policies applied
after 8 Weeks.
by the supply schedule. It is interesting to note that the 10-12 week policy holds slows down
starting week 16. This is because of the need to hold a 3-week reserve for the people who had their
first dose administered starting week 8. In fact, the system starts to hold reserve starting week 15
(8 + 10 − 3: 10-week stretch with 3-week reserve policy starting on week 8) even though it is visibly
noticeable only on week 16.
Figure EC.5 Observing the blocking phenomenon under various aggressive policies.
can deviate from the assumed parameters. Figure EC.6 and EC.7 show that the same degree of
parameter perturbations leads to a wider swing of changes in the holdback policy case compared
with the recommended policy. Specifically, there appear to have more regions where the actual
first-dose demand exceeds the booking limits in the holdback policy case, especially when the
actual response rate is higher than the assumed value. The recommended policy, on the contrary,
can absorb the perturbation better, and in most cases, the perturbation band remains below the
first-dose booking limits.
Figure EC.6 Sensitivity analysis: performance simulation of the different policies under varying arrival distribu-
tions. The red (blue) solid curve shows the cumulative number of first-dose booking limits under
the Hybrid Policy (holdback policy), and the red (blue) dotted curve is the simulated first dose
vaccinations under the assumed log-normal arrival distribution as planned under the Hybrid Policy
(holdback policy), and the red (blue) shaded area highlights the simulated first dose vaccinations
with 10% perturbation of the arrival pattern under the Hybrid Policy (holdback policy).
Figure EC.7 Sensitivity analysis: performance simulation of the different policies under varying response rates.
The red (blue) solid curve shows the cumulative number of first-dose booking limits under the Hybrid
Policy (holdback policy), and the red (blue) dotted curve is the simulated first dose vaccinations
under the assumed response rate of 80% as planned under the Hybrid Policy (holdback policy), and
the red (blue) shaded area highlights the simulated first dose vaccinations under 10% perturbation
of the response rate under the Hybrid Policy (holdback policy).