Professional Documents
Culture Documents
VERSION 1.02
STATUS: business review
22-10-2020
2
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
DOCUMENT GOVERNANCE
VERSION CONTROL
RACI MATRIX
Definitions:
Responsible: The one who performs the work.
Accountable: The one that has the final authority and accountability to a given task
Consulted: The people who provide information for the solution design and with whom there is two-way communication. This is usually
several people, often subject matter experts.
Informed: The people who are kept informed about progress and with whom there is one-way communication. These are people that are
affected by the outcome of the tasks so need to be kept up-to-date.
DISTRIBUTION
REVIEWERS
ABBREVIATIONS
Abbreviation Description
Note: parts of this documents which are market in yellow will be confirmed at a later stage.
5
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
CONTENT
1. PURPOSE
This document outlines the business requirements for “Remote Facilitation Tool for Simulation Exercises, Intra-Action
Reviews and After-Action Reviews” (SimEx & IAR/AAR) and serves as the basis to define project scope and solution
design. It is also intended for potential suppliers to understand the business needs, provide possible proposals and
estimates to deliver a solution.
Estimation should include graphical design (UX/UI), frontend and backend development, unit and functional testing,
security review as well as application publishing.
Vendors are not limited to technology selection and will explain pros and cons of chosen technology.
2. BACKGROUND
As part of WHO’s strategy to improve preparedness to health emergencies, the organization has embarked on a
progressive simulation and review programme to help countries review their current preparedness and response
strategies to public health events and emergencies.
Discussion-based simulation exercises (SimEx), such as table-top exercises can work with remote facilitation or self-
facilitation, given that the right tools are in place to support development and implementation of the programmes.
Similarly, many intra-action reviews (IAR) and after-action reviews (AAR) can be conducted remotely, given that the
right tools are in place to provide support. Traditional methods such as having teams of facilitators travel and support
countries is costly, time-consuming and require a large pool of specialists in order to provide sufficient support. In
addition, the current COVID-19 pandemic has further complicated the execution of face-to-face exercises and reviews
with many countries requiring that teams quarantine for extended periods or simply refuse entry to individuals from
certain geographic locations. As such, remote facilitation is an attractive alternative.
2.2 IN SCOPE
1. Planning/Development Software: With this tool a country would be able to easily plan and develop their own
SimEx/IAR/AAR, based on the pre-defined capacities/pillars (scenario for SimEx) and trigger questions, which
than can be implemented and run independently without further support.
This should also include the design and development of Application Programming Interface (API’s) to share
with and consume data from existing applications (like SPH), tools and databases.
2. Remote Facilitation Service: In addition, we would also like to have the option and flexibility to provide
additional facilitation support, when requested or for more complex IAR/AAR/SimEx. This remote facilitation
service would include the ability to run/facilitate IAR/AAR/SimEx and have plenary discussion as well as
break-out discussions in a closed environment, save online notes taken on predesigned templates through
collaborative processes and produce collaborative reports.
2.4 ASSUMPTIONS
1. The tool will be hosted within the WHO secured environment, after passing a standard security test. This
will require users to have a valid/active WHO accounts (WIMS, AZURE) prior to login and contribute
7
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
usage data. An audit log will capture user specific activities, including login, logout, actions as well as IP
addresses.
2. This data will not be shared outside WHO except as anonymised statistics for reporting purposes. Other
data such as usage data, created exercises and reports will only be accessible to WHO for the purpose of
reporting against the IHR mechanism
3. Users will be able to login from remote terminals from anywhere in the world. The system should be
browser based and will not require any custom software to be downloaded and installed
4. The system should be able to be used in low bandwidth environments
5. Users will be able to interact with others on the system, but the use of anonymous accounts will not be
permitted. New users will require a form of verification, preferably automated (such as validation
through an official email address).
6. A system will be in place to enable the exclusion of potentially malicious users. This can be through an
authentication system or an invite facility.
7. The software developer will not have access to user generated content except for trouble shooting and
debugging purposes. If access is required for development, this will be stipulated, and all data will
remain confidential
8. The vendor will transfer ownership rights to WHO. The vendor will not rent, lease, lend or host products
or service deliverables. The vendor will provide a copy of the applicable product, source code,
documentation and/or any proper instruction, materials and handover to WHO support team for further
update and/or modification.
8
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
The exercise programme guide has been developed in order to enable member states, WHO country offices and
partners to plan and execute simple exercises to test specific core capacities outlined in the International Health
Regulations (2005). The guide is designed to be modular and to enable the organisers of the exercise to choose
specific elements that they wish to test based on their own needs which have been previously identified. While it is
possible for organisers to use the entire guide to `stress test` all 13 core capacities this is not always necessary and
therefore organisers can select one or multiple core capacities based on identified needs. Testing all core capacities
in one exercise will be very time consuming and may need to a disjointed outcome so selecting between one and
three core capacities is seen as an ideal approach.
3.1 METHODOLOGY
Before developing any exercise and setting a scenario it is important that the purpose, scope and objectives are clearly
defined. This will help organisers select the core capacity that is to be tested.
A purpose statement is provided with each core capacity that can be incorporated into your purpose
statement if undertaking more than one core capacity.
1
https://www.who.int/ihr/publications/WHO-WHE-CPI-2018.51/en/
2
https://www.who.int/ihr/publications/WHO_HSE_GCR_2015.13/en/
9
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
2. Scope:
This describes how broadly the exercise encompasses the thematic areas of the core capacities. For
instance, the exercise may be limited to one thematic area and the small team responsible for core
implementation, for example a small team responsible for specific elements of risk communication (Core
Capacity 10), or technical personnel working in one specific laboratory (Core Capacity Five). Alternatively,
the exercise may be larger and involve multiple sectors such as testing Zoonotic Outbreaks (Core Capacity
Three) while also testing national Legislation (Core Capacity One).
By determining the scope of the exercise, the organiser will be able to select the relevant core capacities and
develop the exercise accordingly by selecting the modules enclosed in this guide.
3. Objectives:
Objectives are critical to the design of the exercise as this determines the criteria and ultimately what will be
tested and how. The objectives form the basis of your expected results – what you want to achieve through
the exercise programme.
With each core capacity there is a detailed objective statement that details what is expected when testing
that capacity and is based on the evaluation criteria. Read these and make sure that they meet your needs.
4. Evaluation
Short templates called Participant objective based evaluation templates are included to assist with measuring
the extent to which IHR core capacities have been performed and is largely concerned with the projected
benefits and results of the SimEx.
The evaluation templates should be filled in by all participants immediately after the session and participants
should work to identify the challenges and best practices identified at the end of the debrief session for a
SimEx.
The evaluation templates measures attainment of IHR core capacities, with specific qualitative ratings that
assess the extent to which the capacity performed.
Different definitions of rating levels are provided to guide countries on how to identify areas that require
improvement, and how to acknowledge areas that are strengths. The specific objectives column in this
template contains illustrative examples that need to be adjusted according to the specific objectives of each
AAR/SimEx.
Participants provide assessments of the performance of those objectives, based on the scale below (P, S, M,
U)3.
3
P: Performed without challenge; S: Performed with some challenges; M: Performed with majot challenges; U: Unable
to be performed
10
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
STEP ONE: AGREE PURPOSE, SCOPE, AND OBJECTIVES AND SELECT CORE CAPACITIES TO TEST
1. A short purpose statement is found with each core capacity 4. When choosing more than one core capacity
modify the purpose accordingly but try to keep this consistent with the original purpose
2. Concise Scope. Try to keep the scope short and easy to understand. A short, concise scope is better than a
long, detailed piece.
3. Detailed Objectives. When testing more than one core capacity, use all the objectives without modification
as these refer to the evaluation criteria. This can be highly detailed
4. Select Core Capacities based on this guide and the IHR 2005
Purpose
Purpose of this Core Capacity
Support preparedness and early warning through operational coordination in preparedness, is found here.
planning, surveillance and response by monitoring and reporting incidents at the animal-
human interface. For example, diseases circulating in animals that may not be known
zoonoses but have characteristics that strongly suggest some potential zoonotic threat in the
future requiring a multisectoral assessment of potential zoonotic risk.
Scope
Ensure that mechanisms and documented procedures among all relevant sectors, particularly
those responsible for human health and animal health, are in place to ensure that operational Scope of this Core Capacity is
coordination in preparedness, planning, surveillance and response for zoonotic diseases and found here.
other health events existing or emerging at the human–animal interface, functional
collaboration, and taking a multisectoral One Health approach, are being addressed.
Objectives
C3.1 Collaborative effort on activities to address zoonoses
Objectives of this Core
1. Demonstrate that animal and public health sectors are able to work effectively Capacity is found here.
together at all necessary levels and agreed on collaborative efforts to prevent,
detect and respond to priority zoonoses.
STEP TWO: SELECT THE SCENARIO FROM THE CORE CAPACITIES BEING TESTED AND ARRANGE
THEM INTO A COHERENT STORY.
1. The scenario is only developed once the core capacities have been agreed and the purpose, scope and
objectives have been decided
2. In simple exercises which involve only one core capacity the scenario may be very simple. Do not try to
overcomplicate the scenario with too many variables
3. The scenario simply tells a story that participants can follow and includes timelines and details of what is
happening
4. A sample scenario is provided with each core capacity and this should be adapted to meet the needs of
the exercise
4
Refer to Annex A for the full “List of Core Capacities and their Criteria”
11
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
Inject C3.1: Describe your country’s health emergency. The current situation involves a zoonotic
outbreak that poses an immediate threat to human health and society. The outbreak is affecting livestock
which has an immediate effect on livelihoods and income security. The disease easily spreads to humans
causing a serious, possibly fatal illness (as a guide think of diseases such as COVID-19, Rift Valley Fever,
Middle East Respiratory Syndrome (MERS) or Bovine spongiform encephalopathy (BSE)). You can use a
new fictitious emerging illness.
a. Your country is faced with a major health incident across the animal and human sectors
b. Health and livelihoods are likely to be heavily impacted
Additional Resources:
WHO has multiple resources for those building a scenario from their chosen core capacities. These take the form of
graphics, short video new clips and background stories through to completed exercises that can serve as a useful
guide. Resources are found here:
https://extranet.who.int/sph/simulation-exercise
Resources also include an online learning course to help people new to simulation management get started
STEP THREE: POSE QUESTIONS TO THE PARTICIPANTS USING THE QU ESTIONS SHEET
1. Use the questions detailed in the Core Capacity being tested as these relate to your evaluation process
2. You can ask additional questions to tunnel down to specific areas – don’t be too constrained by the
‘official’ questions.
3. Remember the aim of the exercises is to test the objectives and to get the best out of your teams. This
will help you identify issues early
C3.1 Questions
STEP FOUR: EVALUATE THE EXERCISE BASED ON THE EVALUATION CRITERIA AND AGREE AN
ACTION PLAN
1. The evaluation criteria for each core capacity is included within each part of the guidance
2. The WHO uses four step evaluation criteria based on how well the planning structures support the
outcomes.
3. REMEMBER: Exercises are a test of systems, in this case systematic core capacities. Exercises DO NOT
test individual capabilities or the capabilities of individual participants.
4. Exercises are NEVER used for evaluating individual staff
Evaluating C3.1
Evaluation Criteria for each core
a. Clear policies on collaborative activities should be capacity is detailed in the Evaluation
demonstrated section provided
b. Demonstrated working relations or clear terms of
reference exist between the health and animal sectors
c. MoH & MoA Joint risk and hazard analysis of emerging
threats are undertaken
d. There should be a clear risk register of known and
emerging threats that is coordinated between sectors
e. Information management system is in place for the
effective sharing of information
IHR capacity and indicator Specific objectives (examples, adjust accordingly) Select evaluation
rating*
P S M U
Collaborative effort on activities to E.g. Animal and public health sectors were able to
address zoonoses work effectively together at all necessary levels.
The development of a tool to enable the independent planning and development of IAR/AAR & SimEx that are in a
predeveloped format. Specifically, this will be the IAR/AAR Library and the SimEx off the shelf exercise programme
(OTSE).
The IAR/AAR library is structured around pillars or functions to be reviewed. For each of these pillars, the tool will
present a series of predefined questions to the facilitator, with the flexibility to add and revise question according to
the national context. In summary, the following will be included in the IAR/AAR Library tool:
• The ability for facilitators to collaborate on the terms of reference, trigger questions and review criteria.
These criteria will then be used to select for the database and adjust the trigger questions pertinent for
the specific review to enable the construction of a tailor-made process that can be used to facilitate the
specific review. Facilitators will be able to use the trigger questions to stimulate discussions during the
review.
• Select which pillars or functions they require to be reviewed
• A database of generic trigger questions
• The ability to adapt the generic questions add specific custom questions
• The ability to pre-load templates for the collaborative processes during the conduct of the AAR
The SimEx OTSE, will be designed to be a self-paced programme and can be run as a series of modules. Each of the
modules should be designed for stand-alone use or combined with one or more other modules. There will be 13
modules in total which are aligned with the 13 core capacities outlined in the International Health Regulations 2005
(IHR). The software will include a planning tool that will enable the person customizing the exercise to select the
modules needed and to select from a series of questions and evaluation indicators preassigned to the modules as well
as have the ability to customize these questions and evaluation indicators. In addition, the software should be able to
add three to five custom questions and linked evaluation indicators into each module. Once the module, questions
and evaluation criteria have been selected, the scenario will have to be chosen from different pre-developed (video)
scenarios that can be overlayed into the exercise.
The software will then present a simple exercise that the participants will be able to complete. The exercise will be
presented through a web browser with the option to print part or all of the exercise. The person assigned as the
exercise evaluator will be able to fill a form with the evaluation indicators. In summary, the following will be included
in the OTSE tool:
14
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
• Select which elements of the 13 modules available they require (up to a maximum of 5 for OTSE;
unlimited for AAR). To achieve a good result, the organiser will be presented with a help screen to guide
the selection,
• Select the discussion questions and linked evaluation indicators for each module
• Add/revise any questions for each module
• Select the overall scenario
• Identify and invite target audience/key participants and set a day/time for the exercise
• Run & facilitate the exercise
• Enable participants to vote or provide comment from pull down options to provide user feedback for the
evaluation process
• Evaluate and draft the minimum reporting template
• Approval and submitting of the post exercise report
Linked to the planning tools described above, we want a tool that will enable the completion of IAR/AAR & SimEx
where external remote facilitation support is requested. This remote facilitation service would include the ability to
run/facilitate IAR/AAR/SimEx and have plenary discussion as well as break-out discussions in a closed safe
environment, save online notes taken on predesigned templates through collaborative processes and produce
collaborative reports. Participants to the IAR/AAR or SimEx will be able to collaboratively contribute and discuss, even
when teams are remote from each other. The tool will be flexible and with a focus on collaborative decision making.
As part of this work we expect potential providers to give details of the approach that they would recommend.
• A method for sharing whiteboards and interactive frameworks/interactive notetaking tools allowing the
contribution from all participants.
• A reporting tool allowing to share and export in a structured manner the elements of discussion as well
as the filled evaluation forms that will be used for the final reports.
• For the IAR/AAR process, participants will be pre-assigned to specific pillars and the tool will have the
ability to alternate plenary discussion as well as break-out discussions by pillars. The sessions will be
structured around 5 steps:
a. What was in place before response?
b. What happened during the response?
c. What went well? What went less why? And why?
d. What can we do to improve next time?
e. The way forward
For each step, the participants will have to be able to collaborate in break-out rooms and/or in plenary using
structured note-taking templates as well as voting system. The tool will enable to export in a structured manner the
elements of discussion.
5. REFERENCE MATERIALS
Sample SimEx:
https://extranet.who.int/sph/simulation-exercise
6. APPLICATION CONCEPT
Submit
18
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
Trigger Trigger
Select pillars to be
questions for questions for
reviewed
AAR IAR
Submit
7. BUSINESS REQUIREMENTS
3 As a system administator, I want to be able to create exercises and facilitate Admin 3-high Admin can also act as a facilitator. All
AAR workshop on behalf of member states, WHO country offices and partners. features available to facilitators are also
available to administrators
4 As a system administrator, I want to be able to see detailed information about Admin 3-high
all exercises and IAR/AAR, along with the progress of actions and calendars
5 As an administrator, I want to be able to use any of the existing exercise Admin 3-high
packages and assign it to facilitators/countries.
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews
Remote Facilitation Tool for Simulation Exercises, Intra-Action Reviews and After-Action Reviews