OLIVER WYMAN

WORKING PAPER
COVID-19 Pandemic Navigator Core Model:
Overview of methodology and use cases

Providing forecasts, scenario analysis and sensitivity testing capabilities

for both Detected and Undetected Cases by region

Epidemiology and physics-based, time dependent and data driven

modeling with verified linkages to government actions, mobility trends
and testing
June 10, 2020
Compared to the version from May 15, 2020, this version expends two appendices for scenarios and
death count model
Please email questions or comments to ugur.koyluoglu@oliverwyman.com.
DISCLAIMER
These materials are Oliver Wyman work product. Oliver Wyman makes no representation or warranty
regarding, and accepts no responsibility for, their accuracy, completeness, or predictive nature. Oliver
Wyman has no duty of care in respect of the materials or their preparation; no obligation to update or
supplement the materials; and no liability whatsoever with respect to the materials, any information or
data contained herein, or any use thereof. Oliver Wyman shall not have any liability to any third party in
respect of this report or any actions taken or decisions made as a consequence of the results, advice or
recommendations set forth herein.
© Oliver Wyman

© Oliver Wyman
ABSTRACT

Our compartmental models for COVID-19 spread are based on a discrete form of nonlinear differential
equations, parameterized with dynamic functions (i.e. time-dependent transition rates) that incorporate: (1)
the modeled effects of physical distancing and government actions, (2) evolving testing practices in terms of
timeliness and scale, (3) the link to tracking and contact tracing, and (4) other region-specific characteristics.
Our formulation draws on core methods from both epidemiology and physics. It is data-driven to characterize
non-linear transition rates and posits underlying inference mechanisms that we have verified against mobility
indices representing physical distancing in mass scale and government response actions. Our models
recalibrate daily with the arrival of new information on COVID-19 (“marked to known”). Detected case counts
are calculated from official records, and adjustments are made as needed in case official records have biases
or errors. Undetected case estimates are based on estimates for infection fatality rates, demographics,
mortality by age group and findings from recent seroprevalence test results and other medical studies. By
design, the calibration of our model uses region specific data, as the differences in the timing and execution
of government restrictions, testing, population density, social behavior norms and public response as well as
re-opening plans are extremely relevant. We would like to emphasize the importance of and the need for
capturing nonlinear transmission rates in a flexible fashion and incorporating a Forward view on a daily basis,
given the early stage of the outbreak and time-wise directionally differing and sheer volume of government
and public response against the spread of the virus.
The Core Model is an integral part of our COVID-19 Pandemic Navigator, which provides both predictive
capabilities for the near-term and the ability to perform rich scenario and sensitivity analysis for the next 12-
24 months. Since early April, we have generated daily near-term forecasts for 40 countries, all 50 U.S. states
and 3,143 U.S. counties and 12 Canadian provinces, which have been published on our web site. We have
also started running 10 “what-if” scenarios that explore how the epidemic could play out over the next two
years, along with a number of sensitivity tests with respect to uncertainty in the variables (e.g. infection
fatality ratio, Undetected to Detected ratio, full, partial and limited-time immunity assumptions, etc.).
This white paper introduces the main features of the Core Model as of May 11, 2020. It does not cover the
extensions in the form of conditional (or interactive) models, for which a select list of use cases is provided at
the end of this paper. The Core Model is currently under use, supporting both the public and the private
sectors. Listed use cases are at different stages of development and use - from design to minimum viable
product to production and use.
The unique features of the Core Model include: (i) provides forecasting, scenario analysis and sensitivity
testing capabilities; (ii) considers Undetected and Detected COVID-19 spread; (iii) is based on regional data;
(iv) applies a flexible approach to capture people’s behavior under both government restrictions an
reopening; (v) is re-calibrated, statistically tested (backtested and out-sample tested) and updated on a daily
basis for each region (“marked to known”); (vi) captures physical distancing and government response; (vii)
incorporates inference relationships for transmission rate and Undetected to Detected ratio as a function of
testing and mobility patterns; (viii) is designed as a decision-support tool for multiple use cases; (ix) has a
modular set up to study stratification strategies within each region, across age groups or across regions; (x)
provides multiple options to forecast near-term Forward time-series; (xi) has been shared with the public
since April 8th, 2020 in a limited fashion; (xii) retains past forecasts to review accuracy and stability; (xiii)
generates and analyzes a spectrum of scenarios for each region daily; (xiv) its extension to stochastic version
is available.

1
COVID-19 PANDEMIC NAVIGATOR CORE MODEL

The main part of our Core Model is a representation of the phenomena with five states for each region,
as summarized in Exhibit 1. It builds on the three states contemplated in the SIR (Susceptible, Infected,
Removed) framework by extending Infected and Removed populations into separate parts: Detected
and Undetected.
Detected cases reflect official reports of confirmed cases, while the Undetected universe includes
asymptomatic cases as well as people who were unable or unwilling to be tested, tested but with false
negatives, or tested but results are still unavailable. All transition rates are explicitly modelled as time-
dependent and considered to evolve equation-free at the outset, i.e. not in an externally prescribed way
and forecastable into the future within stochastic error bounds. Please see Yi-Cheng Chen, Ping-En Lu,
and Cheng-Shang Chang (2020). Ceteris paribus, transmission rate from Susceptible to Infected
(Detected) is expected to decay with lockdowns, however its course during the re-opening depends on
the specifics of the re-opening plans and people’s behavior. Testing frequency and coverage also impact
what is Detected.

Exhibit 1: Five-state nonlinear diffusion framework – flux diagram

Our Core Model represents the transitions between states using discrete time counterparts of nonlinear
differential equations based on the representation below. Critically, unlike traditional SIR models, the
parameters that appear in these equations are dynamic, i.e., time-dependent functions that are not
following any prescribed form and estimated from regional data where regions can represent any
segment with coherent reporting of data.

2
 𝒅𝒅𝒅𝒅(𝒕𝒕) 𝜷𝜷(𝒕𝒕)𝑰𝑰(𝒕𝒕) + 𝝃𝝃(𝒕𝒕)𝑨𝑨(𝒕𝒕)
= −𝑺𝑺(𝒕𝒕)
(1a) 𝒅𝒅𝒅𝒅 𝑵𝑵

𝑑𝑑𝑑𝑑(𝑡𝑡) 𝛽𝛽(𝑡𝑡)𝐼𝐼(𝑡𝑡) + 𝜉𝜉(𝑡𝑡)𝐴𝐴(𝑡𝑡)

(1b) = −𝑆𝑆(𝑡𝑡) (𝑝𝑝(𝑡𝑡) + 1 − 𝑝𝑝(𝑡𝑡))
𝑑𝑑𝑑𝑑 𝑁𝑁

𝑑𝑑𝑑𝑑(𝑡𝑡) 𝛽𝛽(𝑡𝑡)𝐼𝐼(𝑡𝑡) + 𝜉𝜉(𝑡𝑡)𝐴𝐴(𝑡𝑡)

(2) = 𝑆𝑆(𝑡𝑡) 𝑝𝑝(𝑡𝑡) − 𝛾𝛾(𝑡𝑡)𝐼𝐼(𝑡𝑡)
𝑑𝑑𝑑𝑑 𝑁𝑁

𝑑𝑑𝑑𝑑(𝑡𝑡) 𝛽𝛽(𝑡𝑡)𝐼𝐼(𝑡𝑡) + 𝜉𝜉(𝑡𝑡)𝐴𝐴(𝑡𝑡)

(3) = 𝑆𝑆(𝑡𝑡) (1 − 𝑝𝑝(𝑡𝑡)) − 𝜃𝜃(𝑡𝑡)𝐴𝐴(𝑡𝑡)
𝑑𝑑𝑑𝑑 𝑁𝑁

𝑑𝑑𝑑𝑑(𝑡𝑡)
(4) = 𝛾𝛾(𝑡𝑡)𝐼𝐼(𝑡𝑡)
𝑑𝑑𝑑𝑑

𝑑𝑑𝑑𝑑(𝑡𝑡)
(5) = 𝜃𝜃(𝑡𝑡)𝐴𝐴(𝑡𝑡)
𝑑𝑑𝑑𝑑

In the above equations, S(t) is the susceptible population, I(t) and A(t) are the infected populations
that are Detected and Undetected, R(t) and Q(t) are the removed populations that are Detected and
Undetected, respectively. 𝛽𝛽(𝑡𝑡) and 𝛾𝛾(𝑡𝑡) are the transmission rate and removal rate at time t for the
Detected. 𝜉𝜉(𝑡𝑡) and θ(𝑡𝑡) are transmission rate and removal rate at time t for the Undetected. 𝑝𝑝(𝑡𝑡)
denotes instantaneous frequency of the Detected population to the sum of Detected and Undetected,
and it is sensitive to the volume of the asymptomatic population and to testing practices in terms of
availability, scale, timeliness and frequency. A sudden increase in the number of random tests will
capture more new Undetected Infected and increase 𝑝𝑝(𝑡𝑡), decrease 1 − 𝑝𝑝(𝑡𝑡).
N is the total population, so that the five variables representing each state add up to N at all times.

(6) 𝑺𝑺(𝒕𝒕) + 𝑰𝑰(𝒕𝒕) + 𝑹𝑹(𝒕𝒕) + 𝑨𝑨(𝒕𝒕) + 𝑸𝑸(𝒕𝒕) = 𝑵𝑵

We believe that these five compartments capture the essence of the spread in time and establish a
foundation from which we can flexibly extend the Core Model to a number of use cases across time and
regions. For example, simple modifications allow us to incorporate the split between the deaths and
recovered cases under the Removed state to estimate death count projections as explained in Appendix B;
or population dynamics with births and deaths added to or subtracted from the equations; or different
immunity assumptions for recovered patients such as limited time immunity, no immunity, or perpetual
immunity. The model is also flexible to accommodate stratification studies.

While this formulation reduces to Susceptible, Exposed, Infected, and Removed (SEIR) models under
some conditions (and we have considered these conditions in the beginning of our research,
development and calibration), and various other transformations can be used, we prefer the five-state
formulation as it captures the essence of the dynamics, is parsimonious and easy to explain. Most

3
importantly, this formulation provides additional flexibility to identify different transmission and
resolution rates for Detected and Undetected universes and to incorporate different uncertainties in
these universes. From its size to the shape and form of its nonlinear dynamics, there are still many
unknowns about the Undetected universe, specifically driven by asymptomatic cases.

Some models focus solely on death counts which is critical information; however we believe there is
value in capturing all states at all times in the future for various uses cases such as understanding the
costs of quarantines based on Detected Infected cases, planning for hospital capacity based on Detected
Active cases, public education based on Detected New cases, understanding how the total of
Undetected Infected and Undetected Removed cases decreases with more testing, estimating future
average mortality rates, and understanding total Infected as a percent of total population to assess
broader immunity, if it exists. For those that are not interested in Undetected Infected and Undetected
Removed, but who just want to incorporate all Undetected, a simpler formulation is to remove 𝜉𝜉(𝑡𝑡),
θ(𝑡𝑡), 𝐴𝐴(𝑡𝑡), 𝑄𝑄(𝑡𝑡) and 𝑝𝑝(𝑡𝑡) from the formulation, and have three simple equations for SIR for Detected
along with a time-varying Undetected to Detected ratio impacting Susceptibles, similar to time-
dependent functions driving an SEIR model.

Our framework is flexible enough to allow for breaking the population into more homogeneous segments
(e.g., by age) and studying location-based differences – or indeed any other partitioning (and we use
partitioning extensively) to describe the real world better by splitting current heterogenous segments into
more homogeneous segments and analyzing differentiated future strategies (e.g., essential workers / non-
essential workers, age stratification, sector segmentation, geographic segmentation, transmission dynamics
as travel restrictions are set / released, mobility across regions, cities, towns, commuters versus non-
commuters, etc.). We extend the state vector for five coupled differential equations per segment to include
additional couplings across segments, e.g., interactive segments. For example, a “5 times K” number of
coupled differential equations is needed to study K separate and non-interacting partitions. For
example, when we study 40 different countries with border shutdowns, we have 40 different sets of five
equations, each group characterized per region, i.e., 40 x (5)=200. If border shutdowns are released
across 9 countries and there is mobility only across these countries, we have one set of 45 coupled
differential equations where cross region activity is incorporated, and 31 separate sets of 5 equations,
i.e. (9 x 5) + 31 x (5) = 200.

Implementation

Beginning in mid- March 2020, we have implemented a discrete version of this model with daily time
steps for over 40 countries and each of the 50 U.S. States (with an extension to the county-level for the
U.S.) and all provinces of Canada, capturing unique characteristics of each region, such as local
containment measures; testing standards, availability, speed and coverage; and model/parameter
uncertainties. We continue to expand both coverage and granularity all around the world. While most
competing industry models have constant parameters or time varying beta in a specific functional form,
our formulation is free-form for 𝛽𝛽(𝑡𝑡), 𝛾𝛾(𝑡𝑡), 𝜉𝜉(𝑡𝑡), θ(𝑡𝑡) and 𝑝𝑝(𝑡𝑡). We first determine these functions for
each day from the discrete version of our compartmental approach and data to date. We then develop a
view on the forward evolution of these functions to calculate forecasts for each compartment.

4
Parameterization follows a system identification scheme that is updated with new information on a daily
basis to reflect Detected cases (main data source: JHU CSSE) with necessary data cleaning. Estimates of
Undetected cases are in a range, calculated from idealized infection fatality ratio studies, reported
deaths, testing seroprevalence, demographics, and other factors on a weighted basis every day. Herein,
we would like to emphasize that the Undetected to Detected ratio is not a constant and varies with time
for each of Infected and Removed compartments. It can be reduced with increased testing but cannot
be eliminated, since our definition of the Undetected universe includes people who are unable or
unwilling to be tested, false negatives, or tested but have not yet received results. The timeliness of
testing / results as well as the scale of testing and the link to tracking and app-based rapid contact
tracing are highly critical to learn more about the Undetected universe. However, even if 100% of the
population were regularly tested (in scale), but they were tested 4 days after becoming infected and
then their results remained pending for a further 2 days (timeliness), there would still be a large portion
of the infected population that were Undetected at any one time.

Our historical database of time-dependent functions for COVID-19 spread reflects characteristics specific
to each region and constitutes a rich dataset for us to learn from and cross-compare system
identification results and to shape (i) predictive (near-term), (ii) scenario analysis (medium to longer
term) and (iii) sensitivity analysis (ranges for critical assessment) parameters.

For forecasting purposes, we started with reviewing and updating trends in actual 𝛽𝛽(𝑡𝑡) (i.e., we deduce
𝛽𝛽(𝑡𝑡) from COVID-19 data) which, as expected, has been trending downwards for regions where
significant containment efforts were in place and captures inference relationships. As the nonlinear
dynamics quickly evolve in the fight between COVID-19 and humans, and given the unknowns and
uncertainties, we learn as we go as a multi-disciplinary team and mark our model’s calibration to new
observations and findings, especially related to sizing the Undetected world and government plans for
re-opening. In a rapid and too early re-opening, 𝛽𝛽(𝑡𝑡) will trend upwards and our approach captures such
directional changes in strategy. We continuously upgrade our thinking and “test and learn” with various
hypotheses to establish top-down (i.e., macro-level, regional, mass scale) and bottom-up (sector,
geography, age stratification, and all the way up to individual behavior level) relationships between the
transmission rate and inference factors such as mobility indices as a proxy to exposure and contact, and
testing (especially recent testing) as a proxy to information value. Similarly, we related the Undetected
to Detected ratio to testing and confirmations, acknowledging that as the testing and confirmed cases
increase, uncertainty around undetected cases decreases.

At the start, all we had was a historical time series of transmission rates, i.e., 𝛽𝛽(𝑡𝑡) implied from COVID-
19 data. The trends in 𝛽𝛽(𝑡𝑡) through autoregressive fits formed the basis to construct our initial Forward
views. As of early April, we moved our attention to understand inference relationships. We express the
changes in 𝛽𝛽(𝑡𝑡) to changes in government strategies for public health, i.e., imposing / lifting restrictions,
and a continuously learning, consciously avoiding, increasing protecting, and cleverly containing human
factor. In parallel, we have been closely tracking the developments in coverage and frequency of various
forms of testing and test results. All together, our views on the future evolution of 𝛽𝛽(𝑡𝑡), 𝛾𝛾(𝑡𝑡), 𝜉𝜉(𝑡𝑡), θ(𝑡𝑡)
and 𝑝𝑝(𝑡𝑡) have been maturing, and we provide our best estimates based on known information at any
time.

We have case count forecasts for the Detected Universe for up to 10 weeks, supported by an increasing
number of and types of statistical tests, since March 22, 2020. The best estimate results for up to 5

5
weeks have been publicly available since April 8, 2020 at https://pandemicnavigator.oliverwyman.com/.
The granular US version with state and county level detail was made public as of April 13, 2020. These
forecasts start from already-captured nonlinear dynamics to date from time-dependent rate functions
and their Forward projections, based on autoregressions for each region that implicitly incorporate
government and human strategies (e.g., containment strategy or uplifting of restrictions). By design, we
have been extracting regional time series for 𝛾𝛾(𝑡𝑡) and 𝛽𝛽(𝑡𝑡) from discretized equations with anchors for
Susceptibles, Infected and Removed cases in this historical data. Any change to the definition of 𝛾𝛾(𝑡𝑡)
marginally impacts 𝛽𝛽(𝑡𝑡), but not the anchors.

As the pandemic progressed, we recognized that Recovered Cases are not reliably reported in many
regions. People who recover are not re-tested to clear their status. These data omissions generally
overstate the number of Active Cases to date and understate 𝛽𝛽(𝑡𝑡) keeping their product constant. To
correct for the missing Recovered Cases, we updated our calibration for 𝛾𝛾(𝑡𝑡) on May 6, 2020, with an
assumption that cases remain active for an average of 14 days. This 14-day assumption is supported by
data from countries with timely reporting of recoveries as well as the experience of the medical and
scientific communities. This update has now been implemented in our projections for both the historical
and forecast period. As a result, the estimated future Active Cases tend to be lower, and peaks occur
sooner in regions where Recovered Cases were incomplete or not provided in a timely manner. For
countries where Recovered Cases are being tracked and reported in a timely manner, we see minimal
changes. Since the product of Active Cases and 𝛽𝛽(𝑡𝑡) is proportional to New cases, and since new cases
is an anchor in our framework, a change of 5% in Active Cases means a change of 1/0.95 in 𝛽𝛽(𝑡𝑡). Please
note the same logic is used in the forecasts, so this change is in-and-out in the sense that we use 14 days
to calculate 𝛾𝛾(𝑡𝑡) and 𝛽𝛽(𝑡𝑡), and we use the same 14 days logic in our projections consistently.

6
Exhibit 2: Public version – Forecasts for United Kingdom (New, Confirmed, Active case counts,
estimates for Undetected cases and % Total Population Infected)

As a second alternative to autoregressions, and as further elaborated in the inference relationship

section below, at a macro level we have linked transmission rate to the measures of mobility, based on
Google’s Community Mobility Indices available by country and state through structured regressions. We
have also linked government response actions as signals to transmission rate by country. Moreover, we
incorporated testing as another inference factor, and our forecasts under the inference model then
include future plans to improve testing and expected changes in mobility.

Our regressions for each region are made up of multiplicative nonlinear terms. We think of mobility as a
strong proxy to represent exposure and contacts. Testing brings the information value for quarantine
decisions – those who tested positive as well as comprehensive and rapid tracking and tracing of those
in close contact with infected individuals. These factors and free parameters in our regressions jointly
capture the impact of all avoidance, protection, identification, and containment actions on transmission
rate. Mobility indices and government restrictions for large public gatherings are examples of mass
avoidances. Equally importantly, the physical distancing in work and public places and decreasing the
number of contacts in daily life are personal avoidances. Shielding the vulnerable is another form of
avoidance. Allowing young people not living with their parents to return to workplaces early is the
opposite of shielding, however it implies a smaller number of life losses. Frequent, timely, large scale

7
testing and the link to tracking and rapid app-based tracing are public identification and containment
efforts, while wearing masks, better hygiene practices and self-quarantines are examples of personal
efforts to protect and contain, respectively.

Inferenceity Revealed and Backtesting

In our benchmarking, we observed strong inference links that explain the containment strategy
rationally. For example, we compare our core transmission index 𝛽𝛽(𝑡𝑡) inferred from only regional
COVID-19 data with measures of mobility, based on Google’s Community Mobility indices available by
country and state, and government response actions, such as the ones listed in the Oxford COVID-19
Government Response Tracker and Index, which provides a systematic way to track the stringency of
government responses to COVID-19 across countries and time. Under containment efforts, significant
decreases in mobility reduce the number of interactions. The impact is captured in 𝛽𝛽(𝑡𝑡) with a lag
(defined by average number of days for symptoms, testing and test results to arrive). Similarly, more
stringent measures result in a significant reduction in transmission rates over the course of the following
days. Exhibit 3 illustrates the impact of such measures on Italy’s transmission rates with a time lag. Our
tool reveals whether and how much mass physical distancing measures have impacted the spread of the
virus for 40 countries and 50 states. The “Social Distancing Effectiveness” charts have been publicly
available since April 20, 2020 at https://pandemicnavigator.oliverwyman.com/.

8
Exhibit 3: Example of mobility indices versus COVID-19 transmission rate for Italy

OW Covid-19 Transmission Rate Google Mobility Index OW Covid-19 Transmission Rate Oxford University
5-day average % change compared to baseline 5-day average Stringency Index

0.5 0 0.5 100

0.4 -20 0.4 80

0.3 -40 0.3 60

0.2 -60 0.2 40

0.1 -80 0.1 20

0.0 -100 0.0 0

2/28/2020

3/11/2020
3/13/2020
3/15/2020
3/17/2020
3/19/2020
3/21/2020
3/23/2020
3/25/2020
3/27/2020
3/29/2020
3/31/2020
2/28/2020

3/11/2020
3/13/2020
3/15/2020
3/17/2020
3/19/2020
3/21/2020
3/23/2020
3/25/2020
3/27/2020
3/29/2020
3/31/2020

3/1/2020
3/3/2020
3/5/2020
3/7/2020
3/9/2020

4/2/2020
4/4/2020
3/1/2020
3/3/2020
3/5/2020
3/7/2020
3/9/2020

4/2/2020
4/4/2020

OW Covid-19 Transmission Rate (Left Axis) OW Covid-19 Transmission Rate (Left Axis)
Transit Stations Mobility Index Change vis-à-vis Baseline (Right Axis) Oxford University Stringency Index (Right Axis)

After Google started sharing its Google Community Mobility Indices in early April, which track changes in
mobility patterns in places like work locations, grocery stores and parks, and Apple and other firms
followed this, it became possible for us to analyze the potential causative relationship between physical
distancing and COVID-19 spread rate. We have observed that during early periods, when an outbreak is
left uncontained, our index remains high and fairly constant, leading to an exponential growth in total
confirmed cases. But, with the implementation of containment measures, we have observed that our
index steadily decays through time, as long as the containment efforts continue. Selecting different
countries and indices in our tool highlights the very strong correlation between these measures. There is
a striking relationship between the change in mobility patterns and the rate at which the virus spreads,
with a lag (i.e., the impact of changing mobility patterns is reflected in the Transmission Rate about 5 to
10 days later). At the tail of the first outbreak, we expect our index to stabilize at minimum levels and
then slowly change course when regions carefully lift restrictions to reopen the economy. All
comparative curves for mobility indices and government actions are available on our web site.

9
Exhibit 4: Example of mobility indices versus COVID-19 transmission rate for Sweden

One region worth discussing is Sweden. Unlike many other countries, Sweden did not enforce very strict
containment measures at the government level. Despite the lack of such draconian measures, Sweden
was able to reduce its transmission rate quickly and kept it at low, manageable levels. There are multiple
factors that contributed to this success – social and cultural norms that are based on fewer personal
interactions (e.g. higher number of single-occupancy households), adoption of protective measures and
social distancing by people independent of government enforcement actions (please see Exhibit 4), and
the ability to track and trace, even after mobility indices started slowly increasing.

In a later section titled Forward Beta Inference model, we present how we incorporated mobile indices
and testing to generate near term projections for Forward Beta.

Forecasting

The model is “marked to known” on a daily basis and continuously learning and adapting to new
information. We test the stability and accuracy of our forecasts for Detected cases, including trajectory-
based back testing and out-of-sample testing for each region daily. Exhibit 5 shows recent forecasts for
confirmed cases and new cases along with forecasts from the previous 7 days for New York on April 23,
2020. Exhibit 5 (bottom right) also shows out-sample results with April 22, 2020 actuals (dashed line) for
confirmed cases compared to the forecast for the same date, starting from each of the previous 7 days.

10
Exhibit 5: Forecasting and Back testing of Pandemic Navigator: Core Model

400,000 Confirmed Cases Backtests (New York) 250,000 Active Cases Backtests (New York)
T -7
T0
350,000
200,000
300,000
T0
250,000
150,000
Confirmed (Actual) Active Cases (Actual)
200,000
T0 T0
T Minus 1 T Minus 1 T -7
T Minus 2 100,000 T Minus 2
150,000
T Minus 3 T Minus 3
T Minus 4 T Minus 4
100,000 T Minus 5 T Minus 5
T Minus 6 50,000 T Minus 6
T Minus 7 T Minus 7
50,000

- -

14,000 New Cases Backtests (New York) Current Confirmed - Forecast Accuracy (New York)
Prior forecasts of April 23 Confirmed Cases level

12,000
300,000
10,000
250,000
8,000 New Cases (Actual) 200,000
T0
T Minus 1
6,000 150,000
T Minus 2
T Minus 3
T Minus 4 T0 100,000
4,000
T Minus 5
T Minus 6 T -7 50,000
2,000 T Minus 7
-
T Minus 7 T Minus 6 T Minus 5 T Minus 4 T Minus 3 T Minus 2 T Minus 1
-

Forecast Actual

Our projections are granular and allow for sophisticated decision-making. For example, our model for the
United States is at the county level, providing a number of heatmaps. Exhibit 6 shows the number of days
from April 17, 2020 until the county reaches 14-Day downward trajectory in new COVID-19 cases, which is
one of the several conditions considered for re-opening, along with having enough hospital capacity,
sufficient testing available, tracking and tracing quarantining, etc.

11
Exhibit 6: Days from April 17, 2020 until county reaches 14-Day downward trajectory in New COVID-19
cases

We believe forecasting nonlinearity is more important than higher order stochastic terms, i.e. we have to
figure out the forward-looking view of the non-stationary mean function accurately first. Nevertheless, for
forecasting probabilistic bounds, stochastic elements are added into the forward path of transmission rates
to determine probabilistic confidence levels for model output. Our statistical tests reveal that mean
evolution of the nonlinear transition rates and uncertainties around them are non-stationary; moreover,
there are lagged relationships across transition rates even for higher moments. Accordingly, we consider
modeling time-dependent functions 𝛽𝛽(𝑡𝑡), 𝛾𝛾(𝑡𝑡), 𝜉𝜉(𝑡𝑡), θ(𝑡𝑡) and 𝑝𝑝(𝑡𝑡) as a vector of non-stationary coupled
random processes and are continuously improving the stochastic formulation. These random processes are
then embedded as transition rates inside our five compartment model for Monte Carlo simulations. Our
simplest models follow block-bootstrapping that captures historical deviations and auto-correlations by
taking de-trended random blocks from historical data appended to create paths into the future, and more
sophisticated approaches include postulating probability distributions and auto-correlations in time. Please
see the scenario section for sample simulation results.

12
To assess model accuracy, we compare our forecasts for each region with the actual data using mean
absolute percent error (MAPE) which provides easy to explain observations, such as the model is within

X% of actual when predicting Y days out, where . The next exhibit shows MAPE
for various countries.

Exhibit 7: Out-of-sample MAPE calculation for confirmed cases

Select countries, using 7-day window of forecasts from 2020-04-28 to 2020-05-04

Geography 1 day out 2 days out 3 days out 4 days out 5 days out 6 days out 1 week out
United 0.3% 0.7% 1.0% 1.1% 1.2% 1.3% 1.5%
States
United 0.6% 1.3% 1.7% 2.2% 2.8% 3.4% 4.1%
Kingdom
Canada 0.9% 1.8% 2.6% 3.4% 4.3% 4.8% 5.3%

France 0.6% 1.0% 1.4% 1.7% 2.1% 2.2% 2.2%

Germany 0.1% 0.3% 0.3% 0.4% 0.4% 0.4% 0.3%

South Korea 0.0% 0.0% 0.1% 0.1% 0.1% 0.2% 0.3%

Outliers are often explainable due to granularity and data issues. In this case Canadian MAPE is the
largest. It is a big country and requires partitioning. Our province level model for Canada turned out to
be more accurate.

We also track the distribution of the relative error and have been seeing very tight error bounds across
all regions. We believe our flexible parameterization and updated calibration are key to having small
errors.

13
Exhibit 8 is across all regions, which reveals that our over/underestimation bounds grow slowly as
forecasts get further away from jump-off (with the vast majority of estimates clustered tightly around
the actual outcomes).

14
Exhibit 8: Error bounds for confirmed case projection over select timeframes
Relative error of forecasts for 38 large countries. Forecasts produced by our algorithm between
2020-04-25 and 2020-05-01. Exhibit tracks relative error for 1 day through 6 days.

15
Sizing the Undetected Universe

Asymptomatic cases, speed of spread and lack of wide, random and frequent testing in many
geographies limits our understanding of the size of the Undetected population. As summarized in Exhibit
9, there are various approaches we considered to produce best estimates and ranges around these best
estimates for the size of Undetected population:
• Carefully tracked groups: These are subsets of populations where the majority (or full
population) is tested for COVID-19 and tracked over a longer duration. One very commonly
cited group is the Wuhan expatriates returning to their home countries. These individuals
have been subject to extensive testing; their transmission, asymptomatic behavior and
fatality rates were carefully tracked, forming an early view of true infection fatality rate
(fatality ratio among all COVID-19 infections, Detected and Undetected) by age group.
Similar analyses were then performed for Lombardy, Italy and the United States, informing
infection fatality rate estimates.
• Closed natural experiments: Experiences from closed environments, such as cruise ships and
military ships, also provide views on infection transmission and asymptomatic behavior. The
Diamond Princess cruise ship, the USS Theodore Roosevelt aircraft carrier, and the Charles
de Gaulle aircraft carrier are a few examples of these studies. Even though these natural
experiments provide insights to asymptomatic behavior, their skew toward certain
subpopulation (e.g., young and healthy groups on military ships) and wider testing in the
subgroup limit the ability to generalize the observations to broader populations.
• Serological (antibody) analysis: Serological tests detect antibodies present in the blood
when the body is responding to a specific infection, like COVID-19. These tests help identify
people who may have been exposed to the virus or have recovered from a COVID-19
infection. When used on a random and sizeable sample of population, these tests provide
insights on the number of individuals that were infected with COVID-19.

16
 • Other research: Research conducted by a group of academics relying on and improving upon
the approaches listed above.

Exhibit 9: Summary of research on the Undetected COVID-19 population

Research area Undetected / Notes

Detected ratio

Carefully Wuhan expats (cross- Implies that it Infection Fatality Rate available by age, can replicate for
tracked referenced to Diamond varies by country any geography
groups Princess) based on
demographics

10 municipalities in Lombardy, Implies that it Infection Fatality Rate available by age, can replicate for
Italy that experienced the varies by country any geography
initial outbreak based on
demographics

1,368 counties in the United Implies that it Based on information up to April 20, 2020, infection
States varies by state / fatality rates for various counties and states are
county based on provided
demographics

Boston homeless shelter Very high 397 tested, 37% infected, all asymptomatic

New York Pregnancy 6 215 tested, 15% infected, 14% symptomatic

Icelandic Study N/A 9,199 tested, 13% infected

Closed Diamond Princess cruise ship 1.2 3,700 tested, 19% infected, 47% symptomatic
natural
experiments USS Theodore Roosevelt 1.5 4,512 tested, 13% infected, 60% symptomatic
aircraft carrier

Charles de Gaulle aircraft N/A 1,767 tested, 41% infected, asymptomatic unknown
carrier

Serological Stanford Santa Clara county 25-94 ~3,300 tested

(antibody) study
analysis
Gangelt, Germany 6 500 tested, 14% positive (2% officially reported)

USC Los Angeles county study 27 – 55 Random testing

New York State 12 3,000 tested, 13.9% were positive (equivalent to 12:1
Undetected to Detected ratio)

NIH and other antibody studies N/A In the works

around the world

Other Columbia University 10 Series of research based on Wuhan experiment and

various quantitative approaches

John Hopkins ~25 Varied, being updated regularly

17
We closely monitor academic research on this topic and incorporate a weighted average view of reliable
estimates into the calibration of the current volume of the Undetected population in total Infected and
Removed as well as time-dependent forward views, based on testing trends and plans.

Two research articles that form the anchor of our global model are the age-based infection fatality rate
methodologies published by Verity, et al. (2020) and Rinaldi, et al. (2020) in peer-reviewed studies,
incorporating the infection fatality rate with age distribution of each geography and deaths to date
divided by confirmed cases 14 days ago. This approach allows us to estimate “true” infected values and
corresponding instantaneous estimates of the Undetected population by country/region, based on age
distribution specific to the country/region (see Exhibit 10). Subsequent exhibits are based on combined
ranges from these two studies.

Exhibit 10: Undetected to Detected ratio ranges by country – as of May 4th, 2020

​Comparison of Wuhan and Italy study on undetected cases

​Ratio of undetected to detected, as of May 4th, 2020
120
Wuhan study
Italy study
100

80

60

40

20

0
Netherlands
Brazil
Canada

Portugal
Indonesia
Czechia

Italy

Mexico
Ireland

Norway

UK
US
Poland
Chile

Iran
Denmark
Belgium

Finland

Romania
Pakistan
France

Israel

Korea, South

Spain
Australia
Austria

Germany

Malaysia

Russia

Thailand
China

Switzerland
Ecuador

Luxembourg

Turkey
Sweden

When we compare derived Undetected / Detected ratios of each country to tests per one thousand, we
observe that as testing becomes more random and widely available, then the Undetected / Detected
ratio declines for each country across the board, as illustrated in Exhibit 11 for a sample country (The
United Kingdom) and Exhibit 13 across countries.

18
Exhibit 11: Undetected to Detected ratio (mid-point values) over time for United Kingdom – as of May
4th, 2020
80 6

70
5
60
4
50

40 3

30
2
20
1
10

0 0
4/1/2020

4/2/2020

4/3/2020

4/4/2020

4/5/2020

4/6/2020

4/7/2020

4/8/2020

4/9/2020
3/26/2020

3/27/2020

3/28/2020

3/29/2020

3/30/2020

3/31/2020

4/10/2020

4/11/2020

4/12/2020

4/13/2020

4/14/2020

4/15/2020

4/16/2020

4/17/2020

4/18/2020

4/19/2020

4/20/2020
Undetected / Detected Ratio (Left Axis) # of Tests per 1K (Right Axis)

Exhibit 12: Undetected to Detected ratio vs. testing availability across countries – as of May 4th, 2020

As of May 4th, 2020

​Tests per 1K of population
50 Israel
Portugal
45 Denmark Ireland
Switzerland Italy
40 Norway
Germany Belgium
35 Austria Spain

Russia
30 Australia
Czechia Canada US
25
Finland
20 Sweden
Turkey Netherlands United Kingdom
15 Korea, South France
Poland
10 Romania
Malaysia Iran
Chile Ecuador
5
Thailand Pakistan Brazil
Indonesia Mexico
-
- 10 20 30 40 50 60 70

Undetected / Detected Ratio – Mid-point

This provides the critical linkage required to estimate the distance to herd immunity. Exhibit 13
illustrates total Infected population as a ratio of total population for select countries along with the
uncertainty bound around it. The mid-point is based on the above-mentioned studies and the range is
the outer range of the two main studies mentioned above.

19
Exhibit 13: Total infected population (Detected + Undetected) as a ratio of total population – countries
Upper level

Mid-point
Lower level
As of May 4th, 2020
10%

9%

8%

7%

6%

5%

4%

3%

2%

1%

0%

Our understanding of the disease is constantly evolving, and more studies and tests are continually
occurring. With the findings of these studies and tests, confidence in estimating the size of the
Undetected population will increase over time. We continuously monitor available research and apply a
weighted average approach among the most reliable set of empirical estimates for each region.

Nevertheless, as explained in the scenario section, we extensively study the sensitivity of the results
with respect to the parameter uncertainty around our best estimates for the Undetected universe.

Near-term Forward Beta Inference Model

Given observations of 𝛽𝛽(𝑡𝑡), we came to a stage where we could reasonably define near-term (e.g. up to
one month) inference relationships between 𝛽𝛽(𝑡𝑡), i.e. observable, and its main drivers in April 2020 to
capture actions that impact transmission rate:
• Avoid: Mass avoidance with reduced mobility (e.g. transit mobility index) as well as personal
avoidance actions (e.g., 6 feet distance, reduction in physical network). The impact of population
density and household density differences between geographies is also reflected here.
• Protect: Usage of masks, gloves, gowns/overcoats, screens as well better hygiene practices. We
expect people to adapt to these quickly, where available, and make more use of these personal
protection actions.
• Identify: Increased testing and quarantines observed in the increased frequency and
extensiveness of testing practices for a given region. The ideal state is to test all as frequently as
possible, to obtain test results very quickly and to use high accuracy tests to detect positives.

20
 • Isolate: Tracking, tracing and isolation (e.g., quarantine) of identified infected population.
Regions that are better at isolation, combined with extensive testing, will be more likely to
manage transmission rates even when social distancing measures are relaxed.
As the pandemic implications evolved around the world, our collective understanding and ability to have
visibility into the future developed. Our modeling of Forward Beta reflects the evolving nature of
availability and quality of information. Public policy will soon directly or implicitly define targets for
effective reproduction rates (likely targeting 0.8 to 1.1 levels), and we will factor such medium-term
targets once available. Our next challenge is to provide near term forecasts based on information from
the battle ground and longer-term scenario analysis to study potential outcomes. We illustrate how we
adapt our calibration and focus as we obtained new information in
Exhibit below.

Exhibit 14: Information timeline dependent modeling

​1. Limited Information ​2. Time series fit ​3. Update daily and test and learn

0.6 0.6 14,000 New Cases Backtests (New York)

0.5 0.5 12,000

0.4 0.4 10,000

0.3 0.3 8,000 New Cases (Actual)

0.2 0.2
T0
T Minus 1
6,000
T Minus 2

0.1 0.1
T Minus 3
4,000 T Minus 4 T0
T Minus 5

0 0
T Minus 6 T -7
2,000 T Minus 7

2/28/2020 3/29/2020 4/28/2020 2/28/2020 3/29/2020 4/28/2020 -

6. Forward Beta: i) Recent history-based,

4. Utilize Mobility Data ​5. Incorporate Government Actions
ii) causal, iii) machine learning
0.6 0 0.6 100
0.5 0.5 0.2
0.4 0.4
0.3 -50 0.3 50
0.2 0.2 0.1
0.1 0.1
0 -100 0 0 0
2/28/2020 3/29/2020 4/28/2020 2/28/2020 3/29/2020 4/28/2020 3/10/2020 5/9/2020 7/8/2020

7. Plausible Opening Scenario ​8. Plausible Summer Scenarios ​9. Long Term Scenarios
0.6 0.6 0.6
0.5 0.5 0.5
0.4 0.4 0.4
0.3 0.3 0.3
0.2 0.2 0.2
0.1 0.1 0.1
0 0 0
Mar-20 Apr-20 May-20 Jun-20 Mar-20 May-20 Jul-20 Sep-20 Mar-20 Mar-21 Mar-22

At the beginning of the Covid-19 outbreak, we could only track Beta behavior over time, and fit time
series to develop a near-term forward view to capture the impact of containment measures. We have
continuously updated our models and calibrated on a daily basis, and the autoregressive approach
worked well for the near-term under the same strategy. After mobile indices became available in early
April, we started to study the relationship between Beta(t) and social distancing, as observed in mobility
indices and as evidenced by government containment actions. Bringing these together, we have three
approaches to estimate forward Beta for the near term, each coming with different strengths and
weaknesses and each updated frequently:
(i) Time series autoregressions focusing on the history-based Beta, which is suitable in
forecasting when similar conditions and the same strategy is in place: For the calibration of

21
 autoregressions, we have started with a full window of information and moved over time to
windows and higher weighting to recent history. That is, we use only relevant data (e.g. last
month time window) and accept that the most recent data is more representative of the
strategy (e.g. the cost function to fit autoregressions is more skewly weighted to recent
history) to mature our Forward views.
(ii) Inference Beta that causatively explains historical Beta as function of social distancing,
testing and personal actions, suitable to capture changing conditions (e.g. when stay-at-
home restrictions are lifted) and to study what-if scenarios: This is explained in great detail
below.
(iii) Machine learning based Beta to reflect non-linearities and a large number of interactions
that are difficult for humans to deduce. This captures a richer set of explanatory variables
across social distancing, government actions, social-economic differences and many other
characteristics.
We assume the autocorrelation approach is well-known for the reader and machine learning provides
power, but we are most interested in understanding and explaining the inference relationships and
expand on our approach to Forward Beta inference model below. As government / state containment
measures and social distancing actions are relaxed, and if no additional action is taken, we expect
Forward Beta to directionally mimic the path in reverse order of what we have seen as containment
measures were put in place. However, protective actions and increased information through testing will
serve to subdue the increase in transmission rates. We believe changes in mobility, after smoothing out
weekends, is a leading indicator, and more recent testing has larger value than total testing done to
date. Bringing these together, we have contemplated the following formulation for Forward Beta for the
near term and calibrated the model by fitting a linear function to the natural logarithm of these terms in
a power setting as follows:
𝛽𝛽(𝑡𝑡) = 𝛽𝛽0 × 𝑇𝑇(𝑡𝑡 − 8)𝑥𝑥 × 𝐸𝐸(𝑡𝑡)𝑦𝑦 × 𝐹𝐹(𝑡𝑡 − 1) 𝑧𝑧
where:
𝛽𝛽(𝑡𝑡): Transmission rate at time t
𝑇𝑇(𝑡𝑡 − 8): 7-day moving average of a mobility index from 8 days ago. This is a leading indicator
where mobility index is taken as a proxy for exposure to COVID-19 and contacts to Infected
𝐸𝐸(𝑡𝑡): Reciprocal of number of tests per 1K of population. More tests generate more information
and better management leading to lower 𝛽𝛽(𝑡𝑡)
𝐹𝐹(𝑡𝑡 − 1): Measures how recent speed of testing vs. recent new cases compare to the practices
since the start of epidemic. We define this parameter as ratio of “relative slope of new cases
over past 7 days to slope of new cases since day 1 as observed at time t-1” to “relative slope of
new tests over past 7 days to slope of new tests since day 1 as observed at t-1”. It is inversely
proportional to negative ratio. Less negatives means higher transmission rate. The one-day
delay breaks circularity, since 𝛽𝛽(𝑡𝑡) is related to New cases (t) by definition, but not to New cases
(t-1).
𝛽𝛽0 : Transmission rate fit at the beginning, before any containment measures were put in place

22
We have built causative models linking COVID-19 spread rate to macro drivers such as mobility and testing
and update them frequently based on both equally weighted data and more weights given to recent
history. As new information arrives and when tracking and tracing indices are available, we will introduce
them to our formulation. For mobility, we use Google’s transit station index (heavy use of transit
transition in large cities), or Apple’s driving index (more rural, less population density), or an average of
Google’s multiple indices for workplaces, retail and transit station, depending on the location. These
indices are created from aggregated, anonymized sets of data from users who have turned on the
Location History setting, which is off by default. The following Exhibit 15 reveals our regressions for
three states in the United States.

Exhibit 15: Inference Beta forecasts – select examples

These relationships allow us to translate re-opening plans to increase in mobility as a proxy to increase in
exposure and contacts, and incorporate testing plans to forecast Beta(t), as illustrated as an example in
Error! Reference source not found.6 for New York. Among all plausible outcomes, we provide only four
“what-if scenarios” on this site
https://pandemicnavigator.oliverwyman.com/forecast?mode=country&region=United States&panel=causal for
interested parties to explore the sensitivities of Confirmed, Active and New Cases with respect to mobility and
testing assumptions.

23
Exhibit 16: What-if analysis on Forward Beta – New York

​Transit mobility ​Transmission Rate Over Time

Reflection of the Plan
0 Active cases as of 05/04: 70,537
-10
-20
-30 0.7
-40
-50
-60 0.6
-70
-80
0.5
3/12/2020
3/19/2020
3/26/2020

4/16/2020
4/23/2020
4/30/2020

5/14/2020
5/21/2020
5/28/2020
4/2/2020
4/9/2020

5/7/2020

0.4
​Testing per 1K
0.3
100
80
0.2
60
40
20 0.1
0
3/12/2020
3/19/2020
3/26/2020

4/16/2020
4/23/2020
4/30/2020

5/14/2020
5/21/2020
5/28/2020
4/2/2020
4/9/2020

5/7/2020

Testing
Testing S1 - Current speed
Testing S2 - Half of current speed
Transmission Rate -- Historical Transmission Rate -- Modeled1
Transmission Rate -- Modeled2

An important input to these calculations is how the Undetected / Detected ratio will evolve for forward-
looking analysis and how we can form consistent and coherent views with consideration for testing
practices in each region.
As explained above, we derive historical, time dependent Undetected / Detected ratios for each region
by applying age-specific infection fatality rates to region and comparing number of deaths to number of
detected infections, with a lag of 14 days. To be able to forecast Undetected / Detected ratios further,
we have modeled this ratio as a function of testing per 1K and the number of confirmed cases for each
region, hence being able to form a view to first bring Undetected ratio to today and then use it on a
forward-looking basis:
𝑈𝑈 𝑈𝑈
(𝑡𝑡) = (0) × 𝐸𝐸(𝑡𝑡)𝑢𝑢 × 𝐶𝐶(𝑡𝑡 − 1)𝑤𝑤
𝐷𝐷 𝐷𝐷
where:
𝑈𝑈
(𝑡𝑡): Undetected to Detected ratio at time t
𝐷𝐷

𝐸𝐸(𝑡𝑡): Reciprocal of number of tests per 1K of population. More tests generate more information
and less unknowns and smaller Undetected volume
𝐶𝐶(𝑡𝑡 − 1): Confirmed (Detected) cases at time t-1. The one-day delay breaks circularity since
𝑈𝑈
𝐶𝐶(𝑡𝑡) is related to 𝐷𝐷 (𝑡𝑡) in our compartmental model

24
The following two exhibits show how the Undetected to Detected ratio is marked to today, and then
forecasted into the future based on confirmed cases and testing, as well as on six sources of uncertainty
that require special attention.

Exhibit 17.A: Undetected to Detected (mid-point estimate) ratio modeling for select regions

Exhibit 177.B: Undetected to Detected approach for factoring uncertainties

25
Exhibit shows four select regions and how well this model explains historical observations, marking
these to today based on the last 14-day path of confirmed and number of tests per 1K.

Exhibit 18: Undetected to Detected (mid-point estimate) ratio modeling for select regions
​Czech Republic ​Indonesia
8 100

80
6
60
4
40
2
20

0 0

​New York ​Maryland

80 60

60
40
40
20
20

0 0

Undetected / Detected Historical

Undetected / Detected Modeled
Undetected / Detected Projections

Leveraging these relationships, we are able to forecast Undetected / Detected ratios for each region in
the near- and longer-term future and use these system-coherent projections in the determination of
Forward Betas and analysis of what-if scenarios.

All of our regression models are reviewed daily and updated frequently, if not daily.

Scenario Analysis
Our framework also includes a deterministic scenario generation and analysis capability to analyze the inter-
connections between health and economic concerns in the medium to long term. In addition, we run
sensitivity analysis with respect to key assumptions. Jointly with the empirical linkages of the transmission
rate to the Google’s Community Mobility Indices and the Oxford University Government Response Tracker
Database and Stringency Index, these capabilities are critical to understand the input/output relationship,
drivers and levers. Insights generated from such studies help to shape public policy planning and execution
to influence the future realization towards the more preferred scenario. Our scenario narratives start with
epidemiology, medical research and advances, as well as learning human behavior, and incorporate
economic pressures and the timing and nature of government response in either imposing or removing
restrictions. These are then converted into rate functions based on learnings from our database, and

26
inference linkages analyzed along with hypotheses on how these linkages might or might not work in the
future. Our equations then generate the trajectories for Infected, Removed and Susceptible populations for
Detected and Undetected. Scenarios studied include different immunity assumptions for recovered patients
such as full immunity, limited-duration immunity or no immunity. Please note the Undetected to Detected
ratio is also time-dependent in our formulation, influenced by testing frequency per population and
confirmed cases to date. Appendix A introduces a standardized suite of scenarios that have been
developed to explore the range of potential outcomes around how the pandemic may evolve over the
next 12-24 months.

Exhibit 19 provides a sample model output for Detected Infected counts. We have Actuals (blue) to April 9,
2020, and this sample scenario starts on April 10, 2020, and follows Detected Infected Cases (orange) under
a set of scenario specific assumptions where a timely exit after the first outbreak is followed by a narrow
band of new cases and active cases stabilize under a reasonable hospital capacity constraint. No seasonality,
no medical breakthroughs, and full immunity is assumed for this scenario. This management continues
successfully until a small second outbreak takes place. There are two other regional and small outbreaks,
mass-scale vaccination emerges in July 2021 and the scenario ends. The grey curve on the left is a sensitivity
test and reveals the impact of uncertainties in the seroprevalence test results from today. As can be seen,
less broad immunity than the orange scenario builds over time. On the right side, we see five stochastic
realizations around this specific scenario’s expectations. There is clearly more uncertainty with time. Peaks
are different, could be larger or smaller, revealing plausible outcomes and urging for continued tight
management with updated information as we go along.

Exhibit 19: Sample scenario output: Active cases

Left graph: Actuals (blue), scenario (orange), sensitivity to the uncertainty in the seroprevalence test result
(grey). Right graph: Five stochastic realizations
Active Confirmed Cases
Active Confirmed Cases

From the very optimistic scenario that we will squash the bug, to the very pessimistic that a region might
not be able to contain a large outbreak in the future, there are multiple foreseeable paths forward
depending on government actions, people’s behavior, epidemiological evolution of COVID-19, medical
breakthroughs, and other key social and economic considerations. In other words, we are in uncharted
territory, and there are a number of unknowns, unknowables and uncertainties – we know that we

27
don’t know, we don’t even know what we don’t know, and we could model using stochastic
formulations but only with care.

In the end, all of these scenarios are not necessarily out-of-our hands or random in nature. There are
avoiding, protective, containing and shielding measures we control to influence the outcome, while
balancing economic and social considerations. Insights from the Pandemic Navigator ultimately serve for
this purpose.

To manage the scenario count, we categorize scenarios into three buckets and study them in detail:

• Timing of lifting physical distancing restrictions – e.g., too soon, timely, late, defined based on
the model’s memory and forecasts
• Expectations after restrictions are removed – e.g., whether people comply with hygiene and
protective measures, whether tracking will be effective, whether there will be regional or larger
outbreaks, whether parents will keep their children at home and home school them even
though schools are open, etc.
• Longer term considerations for future outbreaks and responses – e.g., frequency, timing and
duration of future lock-downs

We supplement scenario analysis with sensitivity tests with respect to key scenario assumptions. While
studying scenario sets and sensitivity analysis, we benefit from the descriptive nature of the model – it is
information-rich yet parsimonious – allowing us to work back from the decisions we want to support to
the information and sensitivity analysis needed to the underlying model assumptions and logic. Again,
we believe that insights generated from such studies can help to shape the public policy planning and
execution to influence the future realization towards more preferred scenarios.

We are running a series of insights from the Navigator to help decision-makers respond throughout the
crisis. The first in the series, COVID-19 Pandemic Navigator: Observe, Orient, Decide, Act outlines how
the Pandemic Navigator can help policymakers and business leaders make informed decisions by
analyzing the impact of potential scenarios. The paper is available here as of April 20, 2020:
https://www.oliverwyman.com/our-expertise/insights/2020/apr/the-covid-19-oliver-wyman-pandemic-
navigator-insight-number-one-.html.

Similarly, we have shared our thoughts about the business implications of the Pandemic Navigator here:
https://www.oliverwyman.com/our-expertise/insights/2020/apr/getting-back-to-work-responsibly.html

EXTENSIONS OF THE CORE MODEL TO VARIOUS USE CASES

As a consulting firm operating in multiple industries and geographies, we have been using the Pandemic
Navigator and extending the formulation to a number of use cases. Forecasts for the near term, scenario
analysis for the medium term and sensitivity tests with respect to key assumptions together support a
number of decisions until a vaccination or other medical breakthroughs emerge. Exhibit 20 provides an
overview of use cases in the public and private sector.

28
Exhibit 20: Pandemic Navigator is designed to support a number of use cases and decisions

Most of the use cases implemented or under development are conditional models and include the
following:

1. Return to office/work planning

2. Government response strategies
3. Cash-flow forecasting
4. Strategic planning and re-baseline budgeting
5. Hospital capacity management
6. Mortality estimates
7. Segmentation and partitioning (essential workers, non-essential workers, age stratification,
commuters versus non-commuters, etc.)
8. Linkage to macroeconomic indicators such as Unemployment Rate and GDP
9. Sector specific cash-flows
10. Company-specific cash-flows
11. Commercial credit losses of a bank
12. Consumer credit losses of a bank
13. Credit migration
14. P&C, Life, and Health insurance risk exposures
15. Collections and restructuring
16. Liquidity planning
17. Capital planning
18. Cost management
19. Changes in customer behavior
20. Policy choices and calibration for re-opening or stringent response for a future outbreak
21. Industry specific models
22. Air travel demand

29
 23. Supply chain breakages
24. Partitioning by age groups, essential/non-essential workers
25. Linkage to Google’s Community Mobility Indices
26. Linkage to Oxford’s Government Response Stringency Index
27. Linkage to Apple’s mobility indices

These extensions and linkages follow either differential or discrete formulations, with the necessary lags.

30
KEY REFERENCES

We appreciate learning from thousands of books, articles and news, and cite only key references and
data sources below:
Yi-Cheng Chen, Ping-En Lu, and Cheng-Shang Chang (2020). “A time-dependent SIR model formulation
for COVID-19,” Institute of Communications Engineering, National Tsing Hua University, Taiwan, R.O.C.
https://arxiv.org/pdf/2003.00122.pdf
Robert Verity, Lucy C Okell, Ilaria Dorigatti, Peter Winskill, Charles Whittaker, Natsuko Imai, Gina
Cuomo-Dannenburg, Hayley Thompson, Patrick G T Walker, Han Fu, Amy Dighe, Jamie T Griffin, Marc
Baguelin, Sangeeta Bhatia, Adhiratha Boonyasiri, Anne Cori, Zulma Cucunubá, Rich FitzJohn, Katy
Gaythorpe, Will Green, Arran Hamlet, Wes Hinsley, Daniel Laydon, Gemma Nedjati-Gilani, Prof Steven
Riley, Sabine van Elsland, Erik Volz, Haowei Wang, Yuanrong Wang, Xiaoyue Xi, Christl A Donnelly, Azra C
Ghani, Neil M Ferguson (2020). “Estimates of the severity of coronavirus disease 2019: a model-based
analysis.” https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(20)30243-7/fulltext

Ginaluca Rinaldi, Matteo Paradisi (2020). “An empirical estimate of the infection fatality rate of COVID-
19 from the first Italian outbreak.”
https://www.medrxiv.org/content/10.1101/2020.04.18.20070912v1.full.pdf
JHU CSSE - Johns Hopkins University Center for Systems Science and Engineering.
https://coronavirus.jhu.edu/data
Google COVID-19 Community Mobility Indices. https://www.google.com/covid19/mobility/
Anna Petherick, Thomas Hale, Toby Phillips and Samuel Webster (2020). “Variation in government
responses to COVID-19,” Blatnik School Working Paper.
https://www.bsg.ox.ac.uk/research/publications/variation-government-responses-covid-19
UN DESA, Population Division (2019). World Population Prospects 2019, Online Edition. Rev. 1

31
APPENDIX A. OLIVER WYMAN PANDEMIC NAVIGATOR: STANDARDIZED
SCENARIOS

This Appendix introduces our scenario design process and offers a standardized suite of what-if
scenarios that have been developed to explore the range of potential outcomes around how the
pandemic may evolve over the next 12-24 months. Each scenario arms end users and decision makers
with information necessary to plan for different eventualities and chart the course ahead.
The standardized set explained in this Appendix reflects our thinking as of April/May 2020. We expect to
make adjustments to the standardized scenario set with the new learnings over time.
Scenario design process
COVID-19 spread is a non-stationary process. Developing scenarios for future COVID-19 spread requires
a thorough understanding of the complex and non-stationary interactions between government policy
and public health tools (diagnostic and serology testing, contact tracing with selective quarantine,
national surveillance system, etc.); private sector and individual responses; local factors such as
population density; innovations and developments in relevant medical fields on vaccination and
therapeutic breakthroughs; and the characteristics of the virus itself. We consider all of these factors to
inform our approach to scenario creation and reflect empirical learnings from the history of the first
wave, such as around levels and trends in average transmission rates across regions, as well as expert
judgment as to how these factors could change going foward, especially as it relates to the impact of
potential major policy actions driven by health or economy concerns in the future.
The first outbreak has been controlled in most regions of the world with increasingly stringent
containment and suppression actions directed by the government and executed by the general
population. Given deep health concerns, stringent containment and suppression actions were needed
and had to be rushed. The tools used were blunt instruments and the results inevitably suboptimal in
terms of incurred economic and social costs (for example, the significant increase in unemployment in
many countries). With economic pressures mounting, the blunt lockdown period for the first outbreak
can now be said to be coming to an end, and we have structured a standard set of descriptive scenarios
around three time periods within the pandemic’s progression:
• Re-opening (May-Jun): Focuses on the immediate decision around when to open the economy
and how fast to do so, and informs the phased lifting of restrictions in a way that accounts for
transmission rates as well as the count of Active Cases.
• Transition (Jul-Sep): Focuses on the transition to a “new normal” over the summer months and
the ability and willingness of the population and its leaders to sustain some minimum set of
containment measures.
• Bumpy Ride (Oct onwards): Focuses on the long haul of suppression and the ability to ramp up
testing and tracing and other non-invasive suppression mechanisms.
For each time period, we examine multiple paths, each with a distinct pattern of events and associated
outcomes in order to explore the key uncertainties and associated decisions that will shape the future
course of the pandemic. Each scenario starts with a narrative describing key actions (such as the level of
testing or the intensity of preventative measures), incorporates quantitative estimates of transmission

32
rates and New and Active case counts for Detected and Undetected, and is applied at the state or
country level.
A summary of the different paths is presented in the exhibit below.

Exhibit A.1 Paths for Opening, Transition, and Bumpy ride

1 2 3
Opening Transition Bumpy Ride
(May - Jun) (Jul - Sep) (Oct onwards)
Too fast / Too Soon Smart Governance and People Comply Managed Target
Open before active case load has dropped Quick ramp up in testing, tracking and Sustained investment in public health
and before public health infrastructure is in tracing; people comply with restrictions infrastructure; minimally invasive
place; rapid rise in new cases prompts a (e.g. no large gatherings) and we prevent a restrictions are successful and we keep
quick return to lockdown summer outbreak new cases within a target range without
large outbreaks

Timely exit
Gradual lifting of restrictions, beginning
with regions that are less vulnerable; new
cases are roughly flat within an acceptable
range
Public Health Infrastructure Not Ready
Public health infrastructure cannot be
scaled effectively and people unable or
unwilling to sustain restrictions, leading to
a summer outbreak and subsequent
lockdown
Cyclical Loosen and Tighten
Infrastructure cannot be scaled effectively
and moderate social distancing ineffective;
multiple “peaks” as cases rise and fall in
response to periods of loosening followed
by periods of tightening

Squash the Bug
Blunt restrictions maintained through early
summer in an attempt to “squash the bug”;
new and active cases decrease
substantially
Public Health Infrastructure Not Ready
and Economic Activity Can’t Sustain
Summer outbreak but continued economic
stress makes a lockdown untenable;
transmission rates and new cases grow
rapidly through the summer
Regional Containment
Containment measures largely effective
but cannot prevent localized outbreaks
requiring regional (i.e. limited to individual
cities or states) lockdowns and travel
restrictions

Take our initial opening period as an example. We contemplate different outcomes based on the timing
and staggering of the openings. To minimize our scenario set, we consider three variations : (i) a Too
Fast / Too Soon path, under which restrictions are lifted before Active Case counts have dropped and
before public health infrastructure is in place, leading to a rapid spike in New Cases (which could prompt
a return to lockdown); (ii) a Timely Exit path, where restrictions are lifted gradually to manage New and
Active cases within a target range; and (iii) a Squash the Bug path where restrictions are left in place
through the summer in an attempt to bring New and Active Cases down as much and as rapidly as
possible.
We then combine one path from each period to create a small number of end-to-end scenarios across
all periods, which include:
1. Smart but also lucky: Assumes the ability to rapidly scale public health infrastructure and the
ongoing effectiveness of more moderate social distancing measures, thereby allowing New
Cases to be managed within a target range without more invasive restrictions.

33
 2. Plausible but optimistic: Assumes containment measures are broadly effective but not sufficient
to prevent localized outbreaks, thus prompting targeted restrictions (e.g. shelter-in-place) in
certain cities / regions.
3. Plausible but pessimistic: Assumes public health infrastructure cannot be scaled effectively,
leading to a second wave in late summer.
4. Frequent blunt lockdowns: Assumes cases rise and fall in response to periods of loosening and
tightening, with multiple waves.
5. Health crisis: Assumes economic activity cannot sustain another blunt lockdown, leading to an
uncontrolled, widespread outbreak.
6. Seasonality: A variation on Smart but also lucky with strong seasonal effects.
7. Crush the virus maintaining tightest possible grip: Patterned after the Korea experience,
assumes transmission rates remain low and future regional outbreaks are quickly controlled.
In addition to the core scenarios outlined above, which focus on the Detected space, we have developed
a set of supplemental scenarios that focus on both Detected and Undetected cases.
Additionally, we examine the sensitivity of our forecasts to assumptions around the virus itself and the
impact of immunity:
8. Immunity questions: Explores the concerns about partial or time limited immunity.
9. Susceptibility hypotheses: Explores the hypothesis that some individuals may be immune to the
disease without having previously been infected.
10. Mutation questions about COVID-19: Explores a scenario where a mutation of the virus causes
widespread loss of immunity at a future point in time.
The exhibit below summarizes the first five scenarios.

34
Exhibit A.2: End-to-end scenarios

The model incorporates local characteristics and tracks things like the history of what happened during
the first outbreak, regional differences in broad immunity, testing to date, and the size of Undetected to
date. This approach will thus yield different results by region for the same input about future
transmission and resolution rates. For example, the same scenario will most likely lead to very different
results for New York City versus Los Angeles. New York City is the closest to broader immunity (if it
exists) in the United States, given that it had the widest exposure in the first wave.
The above mentioned standardized scenarios are applied consistently to each region, with adjustments
to account for (i) the observed differences in empirically-observed transmission rates across regions; (ii)
the start date for each scenario, which is tailored to reflect the fact that each region is at a different
stage in the outbreak; and (iii) the intensity of the first outbreak, so that future outbreaks reflect both
local and global experiences.
Accounting for the Empirically-Estimated Transmission Rates
The transmission rates applied to each region are adjusted to reflect observed differences in historical
rates from the first wave in order to account for region-specific factors, such as population density and
access to public transportation. The exhibit below summarizes the relative transmission rates for a
selection of US states, expressed as the ratio of the state-specific rate to that of the US as a whole.

35
Exhibit A.3: Transmission Rate Scalars for Select US States

Accounting for Different Stages in the Outbreak (Start Dates of Scenarios)

To account for the fact that different regions are at different stages in the initial outbreak, the scenarios
are applied starting when a region reaches a certain level of Active Cases. For example, scenarios that
assume an early open are applied when active cases are still high (roughly 85% of the peak), while
scenarios that assume a more gradual opening are applied when active cases have declined
substantially.

Accounting for Different Stages in the Outbreak (Intensity of First Outbreak)

The intensity of future outbreaks is calibrated to reflect a mix of region-specific experience and the
global experience, measured in terms of peak active cases as a portion of the population. The exhibit
below summarizes the intensity of the first outbreak for selected countries.

36
Exhibit A.4: Peak Active Cases Relative to Population for Select Countries

Narrative and Results for Core Scenarios

1. Smart but also lucky
The Smart but also lucky scenario assumes a gradual opening of the economy, with restrictions lifted in
a balanced manner only after actiave cases have fallen significantly from the peak. The government
makes significant investments in public health infrastructure allowing New Cases to be rapidly identified,
tracked and traced. The population continues to comply with more moderate yet still very cautious
social distancing measures, which are effective in keeping New and Active Cases within target ranges.
Select results linking major elements of input and output - Transmission Rate and New Cases: Smart
but also lucky scenario

2. Plausible but optimistic

The Plausible but Optimistic scenario unfolds similarly to the Smart but also lucky scenario through the
summer months, but starts to diverge later in the year, perhaps with reduced caution and increased
large scale gatherings. Transmission rates remain low, on average; however, it is not possible to prevent
future outbreaks altogether, though these are more sporadic and localized. We see multiple waves, but

37
future peaks are substantially smaller than the recent peak and blunt restrictions such as lockdowns are
only necessary for targeted cities or regions (not country-wide).
Select results linking major elements of input and output - Transmission Rate and New Cases:
Plausible but Optimistic scenario for the United States

3. Plausible but pessimistic

The Plausible but Pessimistic scenario starts the same as the Smart and Also Lucky scenario. However,
the government is not able to effectively scale public health infrastructure rapidly post-open, and
further, the public grows weary of social distancing and other restrictions. As a result, transmission
rates and New Cases grow through the summer until they reach levels observed during the first wave,
prompting policymakers to institute another wave of lockdowns to get the outbreak under control.
Renewed focus, investment, and public vigilance in response to the second outbreak are effective, and
the scenario continues similar to the Plausible but Optimistic scenario, with regional outbreaks.
Select results linking major elements of input and output - Transmission Rate and New Cases:
Plausible but Pessimistic scenario for the United States

4. Frequent blunt lockdowns

The Frequent, Blunt Lockdowns scenario assumes an early, rapid and widespread opening of the
economy before active cases have dropped sufficiently and before the public health infrastructure is in
place to prevent future outbreaks. As a result, transmission rates rise rapidly and new cases spike to
levels above what was observed during the current wave. Policymakers act to reinstate restrictions in
order to bring active cases back down to manageable levels.
Testing, tracking and tracing cannot be scaled effectively or is not accepted by the population, and
transmission rates and New Cases grow again during the first half of the summer, prompting another set

38
of lockdowns to control the outbreak. Less invasive measures prove to be ineffective in preventing
widespread outbreaks. As a result, multiple peaks are experienced, some of similar or greater
magnitude to the current wave, as transmission rates rise and fall in response to periods of loosening
followed by periods of tightening.
Select results linking major elements of input and output - Transmission Rate and New Cases:
Frequent blunt lockdown scenario for the United States

5. Health crisis
The Health Crisis scenario starts the same as the Frequent, Blunt Lockdowns scenario, with a rapid
opening followed by a reinstatement of restrictions. Active Cases again rise during the first half of the
summer; however, in this scenario, another round of lockdowns is not feasible from an economic
perspective, stymieing further action. New cases grow rapidly at rates and to levels far exceeding those
observed during the current wave. Cases eventually peak and start to decline as herd immunity is
reached.
Select results linking major elements of input and output - Transmission Rate and New Cases: Health
crisis scenario for the United States

6. Seasonality
Epidemiologists studying the virus have not concluded definitively whether transmissibility of COVID-19
is dependent on environmental factors such as climate (i.e. seasonality). Accordingly, we study a
variation on the Smart and Also Lucky scenario where a pronounced seasonal effect is observed (a
roughly 20% increase in transmission rates in the winter months and a 20% decrease in summer
months).
Select results linking major elements of input and output - Transmission Rate and New Cases:
Seasonality scenario for the United States

39
7. Crush the virus maintaining tightest possible grip (e.g. South Korea)
The crush the virus maintaining tightest possible grip scenario is patterned after the experience of South
Korea, and it assumes a rapid and widespread build out of the public health infrastructure and a
prolonged period of restrictions.. As a result, transmission rates and active cases decline substantially,
and future outbreaks are quickly identified and isolated.
Select results linking major elements of input and output - Transmission Rate and New Cases: Crush
the virus maintaining tightest possible grip scenario for the United States

For the same input, our scenarios generate different output by region.

Exhibit A.5: Active Cases – Smart and Also Lucky, Plausible but Optimistic, Plausible but Pessimistic,
Frequent Blunt Lockdowns
California

40
Texas

Minnesota

New York

Supplemental Scenarios that Examine the Detected and Undetected Space

The initial development of the scenarios was calibrated in the Detected space, with future peaks sized
based on those observed in the initial outbreak, as it is assumed that policy makers will look at the
Detected universe when making decisions such as when to initiate future lockdowns. However, scenario
modeling needs to incorporate the assumption of a time-varying Undetected to Detected ratio in which,
as testing and tracing capabilities increase, the ratio of Undetected cases to Detected cases decreases.
We developed a supplemental set of the core scenarios that embed the assumption that future
outbreaks will be of a similar magnitude as the experience in both the Detected and Undetected space.
The scenario choice will depend on the level of understanding policy makers have on the effect of
Undetected cases in the initial wave, and whether policy makers will enact lockdowns when reported
(Detected) cases reach a certain level, or whether it requires the average severity of the crisis (in which
significantly more Confirmed Cases are reported) to reach similar levels as the initial outbreak for policy
makers to respond.
The exhibit below compares our core Frequent Blunt Lockdowns to the corresponding supplemental
scenario. The supplemental scenarios examine the potential outcomes calibrated to both Detected and
Undetected cases, resulting in reported (Detected) cases tripling (and Undetected to Detected ratio
decreasing) in comparison to the initial outbreak but maintaining a similar level of average severity.

41
Exhibit A.6: Core Frequent Blunt Lockdowns Scenario compared to Supplemental Frequent Blunt
Lockdowns Scenario

Sensitivity of Assumptions on the Virus

The immunity characteristics of Covid-19 are still unknown and have far-reaching implications for the
future course of the pandemic. The base model does not currently contemplate transitions between
Removed and Susceptible, embedding the implicit assumption that people who have recovered from
COVID-19 cannot be re-infected in the future. However, medical research related to the immunity
characteristics of COVID-19 is inconclusive. The model also assumes that the entire population is
susceptible to the virus (excluding those that have already been infected). In fact, actual case data
suggests certain groups (e.g. children) may not be susceptible. In order to understand the impact these
assumptions have on forecasts and consequent potential policy implications, the scenarios were tested
under two conditions (i) the loss of immunity (e.g. infected individuals re-enter the susceptible
population after recovering); and (ii) partial susceptibility (e.g. reducing the susceptible population).

8. Loss of Immunity
We have examined alternative scenarios where recovered individuals re-enter the susceptible
population after some period of time. In our base scenarios, the susceptible population is reduced by
Confirmed Cases, which embeds the implicit assumption that individuals cannot be infected more than
once (no loss of immunity). In our sensitivity scenarios for loss of immunity, recovered individuals return
to the susceptible population after a period of time (e.g. 90 days). This embeds the assumption that
immunity is only temporary.
All else being equal, temporary immunity will increase the severity of future waves, particularly in
scenarios where we go for herd immunity. The loss of immunity leads to several waves of outbreaks as
the susceptible population grows again over time.

42
Exhibit A.7: Immunity Assumptions

43
APPENDIX B. MODELING COVID-19 RELATED DEATHS

Among measurable outcomes of the COVID-19 pandemic, COVID-19 related deaths are a key metric of
interest. Our compartmental model for forecasting COVID-19 related deaths, described in this Appendix,
is an extension of our Core Model for COVID-19 spread.
Aligned with the overall modeling approach of the Pandemic Navigator, we model deaths using the
discrete form of the following differential equation:

𝑑𝑑𝑑𝑑(𝑡𝑡)
= 𝜔𝜔(𝑡𝑡)𝐼𝐼(𝑡𝑡)
𝑑𝑑𝑑𝑑

where 𝐷𝐷(𝑡𝑡) represents cumulative COVID-19 related deaths, 𝐼𝐼(𝑡𝑡) is the infected population that is
Detected (following the definition in the Pandemic Navigator). 𝜔𝜔(𝑡𝑡) is the rate at which some of those
infected individuals die. Importantly, the 𝜔𝜔(𝑡𝑡) is modeled as time-dependent and not following a
prescribed function.
The exhibit below provides examples of the historically-observed values of 𝜔𝜔(𝑡𝑡) for two states:

Exhibit B.1: 𝝎𝝎(𝒕𝒕) parameter time series for New York (left) and California (right)

Empirical evidence demonstrates differences in observed death rates over time, for example, as
expanded testing allows us to capture milder cases (which are less likely to result in death), and across
regions, for example, due to differences in testing or in demographics. Death records also reveal
variations during the week. The exhibit below presents the average 𝜔𝜔(𝑡𝑡) parameter for a one-month
period (ending May 12) for U.S. States.

44
Exhibit B.2: Daily average of 𝝎𝝎(𝒕𝒕) by state (4/13 – 5/12); Top-10 states labeled (by Cumulative Deaths
to 5/12)

Implementation
Actual values for 𝜔𝜔(𝑡𝑡) are derived directly from discretized equations and the historical data for each
region, while a forward view is developed by extrapolating recent trends through 7-day autoregressive
fits with more weight given to recent history to capture the variations during the week and trends over
time. The exhibit below shows a sample of actual and fitted 𝜔𝜔(𝑡𝑡) for two states, with an in-sample fit
calibrated using data through 5/24.

Exhibit B.3: 𝝎𝝎(𝒕𝒕) Actuals and Fits for New York (left) and California (right)

Forecasts of 𝜔𝜔(𝑡𝑡) based on times series analysis are combined with forecasts of Active Cases from the
Core Model of Pandemic Navigator to produce forecasts for cumulative and incident deaths.

45
In addition to point estimates, we model uncertainty in our forecasts arising from the following:
1) Uncertainty in the 𝜔𝜔(𝑡𝑡), which drives the proportion of Active Cases resulting in death, and
2) Uncertainty in the transmission rate β(t), which drives forecasts of daily New and Active Cases.
The error terms are characterized by stochastic modelling and 10,000 simulations. 5 simulations of
forecasts for w, beta, incident deaths, and cumulative deaths are plotted below for New York State, as
of 5/24.

Exhibit B.4: Actuals and 5 Simulations of 𝝎𝝎(𝒕𝒕), β(t), Incident Deaths, and Cumulative Deaths for New
York

The paths are then ranked for each time step to estimate confidence intervals. A sample of this analysis
is shown below for New York, which depicts point estimates, 0.05 quantile, and 0.95 quantile for
cumulative deaths, extending 4 weeks from the 5/24 jump-off point.

46
Exhibit B.5: Actuals and Confidence Levels for Cumulative Deaths in New York

Testing results
In order to assess model performance, out-of-sample backtesting was performed across regions and
time periods to compare model forecasts to actuals. A sample of this testing is shown for New York and
California in the two exhibits below:

Exhibit B.6: Backtesting of Cumulative Deaths using models calibrated through 5/17 for New York
(left) and California (right)

47
Exhibit B.7: Backtesting of Cumulative Deaths using models calibrated through 5/24 for New York
(left) and California (right)

The testing results demonstrate the strong performance of our models on both in- and out-of-sample
fits.

48