AUDITING AGAINST THE COSO ERM 2017

ACUA Annual Conference, Wednesday, September 12, 2018, 11:06 AM

This audit program was crowdsourced during the session at the ACUA annual conference in September
2018 entitled “Auditing Risk Management Using the New 2017 COSO Enterprise Risk Management (ERM)
Framework.” Deena King, CISA, CCEP, and Director of Compliance at Texas Woman's University
facilitated this session. Input on this audit program came from a variety of colleges and universities,
including University of Oregon, Virginia Tech, University of Texas System, Stanford, University of South
Florida, and many others.

Because this audit program came out of a crowdsourced workshop, these notes are similar to what might
come out of a brainstorming session. Therefore, they could use some additional scrutiny against COSO
published materials—especially Enterprise Risk Management – Integrating Strategy and Performance.

Overall, please consider this audit program a tool kit from which you can “pick and choose” which
elements best fit your organization. Most institutions are probably be too small for a full-blown enterprise
risk program as advocated by this ERM guidance provided by COSO.

Acknowledgements: A great deal of thanks goes to the note takers for this session who were Trisha
Burnett, University of Oregon, Virginia Kalil, University of South Florida, and Brian Daniels, Virginia Tech.
This audit program “tool kit” would not be possible without them.

Very Brief Overview of COSO 2017

This is the new ERM model from COSO (2017):

©2017 Committee of Sponsoring Organizations of the Treadway Commission (COSO). Used by permission.
There are 20 risk management principles in the COSO 2017 framework (see below). The following audit
program addresses each of these principles.

©2017 Committee of Sponsoring Organizations of the Treadway Commission (COSO). Used by permission.

In addition, COSO recommends using the new ERM framework in conjunction with the COSO Internal
Controls – Integrated framework (see below). This framework helps understand how control principles
need to penetrate through all layers of an organization.

COSO Sources
Enterprise Risk Management: Integrating with Strategy and Performance - Executive Summary, ©2017,
Committee of Sponsoring Organizations of the Treadway Commission (COSO). Free download.

Internal Control – Integrated Framework, ©2013, Committee of Sponsoring Organizations of the

Treadway Commission (COSO).

Enterprise Risk Management – Integrated Framework, ©2004, Committee of Sponsoring Organizations of

the Treadway Commission (COSO).

Links to all sources can be found at coso.org (“Guidance”). All rights reserved. Used with permission.
Auditing Enterprise Risk Management against the COSO 2017
ERM Principles
There are three sections beneath each principle that inform this audit program:

a. Ideally, what would this COSO principle look like in practice?

b. What documents would an auditor request to validate that practices are in line with COSO
principles?

c. What questions might an auditor ask to validate that practices are in line with COSO
principles?

Note: Some of the items below may actually address multiple principles. It is difficult to have a large
group discussion about risk in one area without drifting into other risk areas. Feel free to move things
around as needed.

Governance & Culture

1) Exercises Board Risk Oversight:
a. Ideally, what would this look like in practice?
∼ Board is trained in ERM
∼ ERM is a specific responsibility listed in the Board charter; is charter reviewed and
updated somewhat regularly?
∼ What is their meeting frequency?

b. Documents?
∼ Charter, meeting minutes, Ethics policy, cultural statements

c. Questions?
∼ Has the Board been trained on ERM oversight?
∼ What does the board see as their responsibility?
∼ Do they understand risk management?
∼ Are they comfortable with the risks at the institution?
∼ Do they have info to make decisions on risk? Happy with frequency?
∼ How are risk decisions considered in other areas?
∼ Risk appetite and tolerance defined?
∼ How is risk impacting the decisions of the board?

2) Establishing Operating Structures

a. Ideally, what would this look like in practice?
∼ There a risk committee of the Board or risk is the responsibility of another Board
committee
∼ There is a risk management department (or there is one somewhere—opinion:
existence matters more than location; as long as risk is covered somewhere)
∼ Position for a qualified risk manager

b. Documents?
∼ All of the above is documented in charters, job descriptions, org charts, reporting
matrix, policies and procedures, etc. Matrix of some kind that shows requirements

c. Questions?
∼ Is there a risk committee or is the responsibility in another committee for
 oversight?
∼ Are you satisfied with the current risk organization?

3) Defines Desired Culture:

a. Ideally, what would this look like in practice?
∼ Two extremes are risk adverse or risk aggressive; defined risk culture needs to be
something in the middle that works with your organization’s strategy
∼ Risk and risk management are addressed in the strategic plan
∼ Acknowledge that culture is a big part of risk management
∼ Tone at the top, mood in the middle, behavior at the bottom
∼ All should line up with the mission, vision, values

b. Documents?
∼ Strategic plan, mission, vision, values, exit surveys; climate surveys; financial
incentives, hotline complaint patterns, etc.

c. Questions?
∼ What is the defined risk culture for this organization?
∼ Has the culture been documented and approved by management and the board?
∼ What are the incentives in the organization?
∼ Do all necessary elements match up?

4) Demonstrates Commitment to Core Values

a. Ideally, what would this look like in practice?
∼ Reflected in the budget
∼ Whistleblower policy
∼ Non-retaliation policy
∼ Types of channels exist
∼ Managing cases properly (prioritize projects according to values)
∼ Process for investigations
∼ Training
∼ Task force reports

b. Documents?
∼ Documents from above

c. Questions?
∼ Conduct an anonymous cultural survey
∼ What actions is management taking to demonstrate commitment to these values?

5) Attracts, Develops, and Retains Capable Individuals

a. Ideally, what would this look like in practice?
∼ You want skilled, capable, credentialed individuals doing the jobs; without this it
puts us at risk
∼ Not run like a 'mom and pop'
∼ Job descriptions require specific certifications and experience
∼ Should have an HR program that recruits, screens, evaluates candidates
∼ Succession planning process, low turnover, annual evaluations

b. Documents?
∼ HR policies and procedures; professional growth procedures; performance
 evaluations; recruiting plans; recruiting files and notes; market surveys; succession
plans

c. Questions?
∼ Do current risk management personnel have adequate skills, certifications, and
experience?
∼ Is there turnover in key positions? If yes, why?
∼ When criteria are met, are there promotions from within?
∼ Any statistics from performance evaluations?
∼ How are we bridging the risk mgt gaps?

Strategy & Objective Setting

6) Analyzes Business Context
a. Ideally, what would this look like in practice?
∼ Benchmarking with peer institutions: Enrollment, retention, performance indicators
for faculty and students, etc.
∼ Where is the money spent?
∼ Legislative impact?
∼ Publicly available risk information—industry specific, when available
∼ External political, economic, social, technology, legal, and environmental
factors are considered
∼ Internal capital, people, process, and technology factors are considered

b. Documents?
∼ See above
∼ Business context report
∼ List or sources used in analysis
∼ Verizon Risk Report (free download)

c. Questions?
∼ How was the business context analyzed?
∼ Were internal and external factors considered (see list above)?
∼ Were the results documented?
∼ What are our competitors/peers doing?
∼ Are you happy?
∼ Meeting expectations?
∼ Resources sufficient?

7) Defines Risk Appetite

a. Ideally, what would this look like in practice?
∼ Risk definition exercises conducted with management to establish limits (tolerance;
acceptance)
∼ Expressions implemented should include target, range, ceiling, and floor as well as
capacity and profile.
∼ A report (articulated risk appetite) comes out with the result of existing risks,
acceptance, tolerance, to present to the Board to drive the setting/evaluating risk
appetite

b. Documents?
∼ See above report
 c. Questions?
∼ Are you satisfied with the current risk appetite (see expressions above) and how it
is defined?
∼ What are the thresholds where the Board gets involved?
∼ What is the tolerance of the board?

8) Evaluates Alternative Strategies

a. Ideally, what would this look like in practice?
∼ Multiple strategies for addressing risk are discussed and evaluated
∼ ERM tabletop exercises could assist with identifying strategies
∼ One valid strategy is risk acceptance

b. Documents?
∼ Documentation showing that multiple strategies have been considered

c. Questions?
∼ Are you happy?
∼ Resources sufficient?
∼ How would a funding decrease impact risk?

9) Formulates Business Objectives

a. Ideally, what would this look like in practice?
∼ Budget and forecasting process
∼ Different levels considered: department, management, and university
∼ Business objectives are aligned with organizational strategy

b. Documents?
∼ Mission, vision, goals, objectives, values
∼ Budget, forecasts; Strategic plan all tie into business objectives
∼ List of business objectives that show how they align with strategy

c. Questions?
∼ Are business objectives defined?
∼ Are these objectives aligned with organizational strategy?
∼ Are you happy?
∼ Resources sufficient?

Performance
Note: At this point, it should be noted that an organization should not begin the following steps
until the previous steps have been completed, at least at a basic level. According to COSO, the
above steps lay the foundation and set the context for enterprise risk identification and
assessment.
10) Identify Risks
a. Ideally, what would this look like in practice?
∼ There is a defined, documented risk universe that impacts performance, strategy,
and business objectives
∼ What could prevent us from achieving objectives?
∼ Brainstorming meetings
∼ KPI'S
∼ Iterative/continuous assessment
 ∼ 1:1 meetings with key management
∼ Peer institute collaboration
∼ Homework/research
∼ Online survey
∼ Any/all of the above
∼ Final result: A risk inventory

b. Documents?
∼ See above; risk register; meeting minutes; survey results; interview notes; financial
reports; risk inventory

c. Questions?
∼ Is there clear understanding how risks emerge?
∼ Is there a repeatable risk identification process?
∼ Who was involved? How often?
∼ Does the process take into consideration an adequate risk universe?
∼ Does the process result in an adequate risk inventory?
11) Assesses Severity of Risk
a. Ideally, what would this look like in practice?
∼ Input into this process will be the risk inventory from 10) above
∼ Very subjective, but can make it objective
∼ Can be measured by financial thresholds
∼ External look at how it impacted others; external environment considered
∼ Reputational risks; health and safety considered
∼ Utilizes impact, likelihood, qualitative, quantitative, and frequency measures
∼ Output from this process will be the same risk inventory with severity added to
each item and a heat map or other depiction or how risks rank on a severity scale

b. Documents?
∼ Risk severity assessment process documents
∼ Risk Report - is there an assessment of severity; does it include scales?
∼ A heat map or other depiction or how risks rank on a severity scale

c. Questions?
∼ Did the process take into consideration necessary measures (see above)?
∼ Have risks been adequately ranked? Is there a report?
∼ Are you happy?
∼ Resources sufficient?
12) Prioritizes Risk - how the university will respond vs. measuring severity
a. Ideally, what would this look like in practice?
∼ Input for this process will be same risk inventory with severity added to each item
∼ This process takes into consideration business strategy, objectives, and appetite;
bias is avoided
∼ Output for this process will be the risk/severity inventory put in priority order

b. Documents?
∼ Risk raking process
∼ Risk/severity inventory ranked by priority

c. Questions?
∼ Are you happy with how risks are ranked?
 ∼ Were business strategy, objectives, and appetite taken into consideration?
∼ Was bias avoided?
13) Implements Risk Responses
a. Ideally, what would this look like in practice?
∼ Input for this process will be the risk/severity inventory put in priority order
∼ A response plan for each prioritized risk which utilizes the following categories:
accept, avoid, pursue, reduce, and share and takes into consideration business
strategy and objectives, priorities, appetite, and severity; also cost/benefit
∼ Evidence the response plan has been implemented
∼ Plans in place - KPI's measuring risk response
∼ Presenting to the Board periodically
∼ Risk champion/risk owner
∼ Policies updated when risks/responses are identified

b. Documents?
∼ Documented risk response designation process
∼ Risk inventory with corresponding response(s); Action plans; reports to board or
mgmt.; team membership; external reviews/audits

c. Questions?
∼ Is the response plan appropriate?
∼ Are we happy with how we plan to respond? responded?
∼ Did we go too far? Not far enough?
∼ Are the risks mitigated?
∼ Has the Board signed off on residual risks?
14) Develops Portfolio View
a. Ideally, what would this look like in practice?
∼ An advanced process
∼ Risk is known from A to Z
∼ Risk identification and responses are in line with company strategy at the entity
level
∼ Risk management is driven by the organization’s strategy and objectives, top down

b. Documents?
∼ Copy of the risk portfolio (or an index)

c. Questions?
∼ Can all risks in the inventory be tied to a strategic imperative?
∼ Are you happy? Resources sufficient?

Review and Revision

15) Assesses Substantial Change

a. Ideally, what would this look like in practice?
∼ Change management at the entity level takes risk management into consideration
(“Substantial change may lead to new or changes risks…”)
∼ Effects of change are evaluated
∼ Post mortems are done after a risk event that reviews responses and their
effectiveness

b. Documents?
 ∼ Entity change management documentation showing risk is considered
∼ Post mortem process/notes

c. Questions?
∼ Was risk taken into consideration when substantial changes occurred in entity
strategic plans?
∼ Are you happy? Resources sufficient?
16) Reviews Risk and Performance
a. Ideally, what would this look like in practice?
∼ Retro reviews
∼ Treat it like an audit (evaluate criteria against existing processes or this COSO
framework)
∼ Periodic sign offs and reviews
∼ Reports to senior management and the board
∼ Monitoring by risk management function of the risk owners
∼ Follow up process

b. Documents?
∼ ERM review process
∼ ERM review results/report

c. Questions?
∼ Are you happy with the risk management process?
∼ Is risk being managed as we intended?
∼ Are gaps closed?
∼ Note: p. 92 in Enterprise Risk Management – Integrating Strategy and Performance
has a list of questions to consider during this review.
17) Pursues Improvement in Enterprise Risk Management
a. Ideally, what would this look like in practice?
∼ Similar to the audit QAR; Quality assessment review done periodically
∼ Where can improvements be made?
∼ Timely/periodic reviews
∼ Training
∼ Updates received
∼ Resources available

b. Documents?
∼ QA process/program
∼ See above - report on QA
∼ Evidence of changes needed and changes made

c. Questions?
∼ Is our ERM working as intended?
∼ Is continuous improvement part of the ERM process?
∼ When was the last ERM program assurance evaluation?
∼ How often is the process evaluated?
∼ Are you happy? Resources sufficient?

Information, Communication, & Reporting

18) Leverages Information and Technology

 a. Ideally, what would this look like in practice?
∼ The risk management process is automated using software designed for ERM
∼ Leverage from feedback, research, benchmarking, newspapers, websites, failures
of other organizations, etc.
∼ Keeping up-to-date with trends, innovations
∼ Training/networking with other risk managers
∼ Certifications
∼ Note: p. 100 in Enterprise Risk Management – Integrating Strategy and
Performance has a good list of internal data sources.

b. Documents?
∼ Prospectus of ERM software used
∼ Sample reports from the ERM tool
∼ Sample of resources used to keep up-to-date with latest ERM trends

c. Questions?
∼ Are you using technology to help manage ERM?
∼ How do you stay up-to-date with ERM innovations?
∼ How often is your ERM training updated?
∼ Are you happy? Resources sufficient?
19) Communicates Risk Information
a. Ideally, what would this look like in practice?
∼ Risk Communication Plan
∼ Everyone who needs risk information gets it in a timely manner
∼ Alert system, if necessary
∼ System of communication is in place and tested periodically
∼ Communication occurs top to bottom - Board, senior mgmt; students; faculty; staff;
stakeholders
∼ Note: p. 104 in Enterprise Risk Management – Integrating Strategy and
Performance has a good list of communication methods

b. Documents?
∼ Communication plan/process
∼ Sample surveys, emails, posters, coasters, pens, swag, etc.
∼ Anything that shows the communication plan/process is used and is working

c. Questions?
∼ How often is risk information communicated and to whom?
∼ Are there committees and teams who meet regularly?
∼ Are you happy? Resources sufficient?
∼ Has risk communication been effective?
20) Reports on Risk, Culture, and Performance
a. Ideally, what would this look like in practice?
∼ Board gets scheduled reports
∼ Senior management gets scheduled reports
∼ Annual risk report
∼ Reports using key indicators
∼ Penn St. is a good example of a cultural turnaround
∼ Risk culture surveys and results that are acted on
∼ Note: p. 106 in Enterprise Risk Management – Integrating Strategy and
Performance has a good list of types of reporting methods
b. Documents?
∼ Reporting schedule
∼ Copies of the above reports and survey results
∼ Evidence of action taken on gaps

c. Questions?
∼ What groups are getting risk reports?
∼ Is the Board informed about risk on a regular basis?
∼ Is senior management part of the risk management process?
∼ Stakeholders?
∼ Is action taken to close gaps?
∼ Are you happy? Resources sufficient?