TMISA12 Ernst & Young2_Layout 1 15/11/2012 12:19 Page 26

Managing Treasury in the New Economy

The Risk in Your

Valuation and Risk
Calculations
by Carel van der Merwe and Jaco van Tonder, Quantitative Advisory Consultants, Ernst & Young

M
odel risk poses an increasingly higher threat due to the ever more
widespread use of models. In essence, mathematical models are used
to try to capture an observed phenomenon. At first this risk might
seem to be of little importance to corporate treasuries; however, it could
possibly be a significant risk that is sometimes overlooked.
An observed phenomenon could be either a past or a future event. A model
attempts to capture these and perform calculations based on a mathematical
approach to produce a certain output. As an example, past events could include
past inflation movements; while future events could include future changes in
interest rates. Outputs could include asset valuations, impairment provisions
and many other parameters used for, amongst other things, business reporting
and decisions. Some items that may constitute both an output of one model,
and the input of a next, like volatility and dividend projections used in
valuations, are also estimated with models. All financial instrument valuations
are performed using models – these are just but a drop in the ocean of the other
possible applications.
Corporate treasuries are mostly exposed to valuation models; however more
treasuries are performing risk analysis internally as well.
Risk analysis, often relying on mathematical models, has in recent years
become an integral part of business and organisational decision-making
processes worldwide. The calculation of risk is in many cases more an art than a

Corporate treasuries are mostly exposed to

valuation models; however more treasuries are
performing risk analysis internally as well.

26 TMI | Special report

TMISA12 Ernst & Young2_Layout 1 15/11/2012 12:19 Page 27
Managing Treasury in the New Economy
science. At the same time, high impact
Figure 1 Fig 1
decisions are often made based on the
output of these models. Judging by the
relative importance of the decisions based
on the results of a risk report, it is advisable
for corporate treasuries to also manage the
risks inherent in the models used for the
measurement of risk.
Model risk is the risk associated with
trying to capture an observed phenomenon
in a mathematical model. The term ‘model’,
for the purpose of this article, can be
defined as the collection of methods
producing quantitative estimates for future,
uncertain quantities, based on mathematical
techniques and assumptions, sometimes
including expert judgment.
Models are by construction simplified
representations of complex phenomena. For
example, what will the payoff of a financial
instrument be in the future? Ultimately,
models are judged by their predictive ability
to capture the relevant features of the
underlying process. Thus, rather than only
looking at potential errors in the construc-
tion of the model (an essential part of the
review process that should not be ignored),
one should also look at potential errors in its
application. Model risk may arise from
incorrect model application, design, imple-
mentation, calibration errors, programming
errors and data problems.
Model risk needs to be managed like any
other risk, and should follow the cycle of
identification, assessment, risk mitigation
and monitoring. This implies the use of the
tools, roles and responsibilities, including the
three lines of defence related to the use of
models being first line (model owners),
second line (model validation) and third line
(internal audit). A proper model risk
management framework could prove helpful
in managing model risk efficiently.
Model risk increases with greater model
complexity, higher uncertainty about inputs
and assumptions, broader use, and larger
potential impact. A proper model risk
management framework should be instilled
with an understanding of the source and
magnitude of model risk. The principle of
‘effective challenge’ of models is used as
guidance for managing model risk. That is,
the critical analysis by objective, informed
parties who can identify model limitations,
including assumptions, and produce
appropriate changes.
TMI | Special report
Model Risk Management Framework
1st Line of Defence
(Model Owners)
Model Lifecycle
Model Review Components
Business Purpose
Model Development
Model Governance
Model Implementation
Model Operations
Ongoing Performance
Monitoring
Change Management
3rd Line of Defence
(Internal Audit)
Model risk is the risk associated with trying to capture an
observed phenomenon in a mathematical model.
Effective challenge depends on a
combination of incentives, competence, and
influence.
Incentives to provide effective challenge
to models are stronger when there is greater
separation between the parties challenging
and the model development process, also
when challenge is supported by well-
designed compensation practices and
corporate culture. Competence is key to
effectiveness, since technical knowledge and
modelling skills are necessary to conduct
appropriate analysis and critique. Finally,
challenge may fail to be effective without
the necessary influence to ensure the
actions are taken to address model issues.
The next section provides an introduction
to a generic model risk management
framework, including the three lines of
defence mentioned above. This will be
followed by a more in-depth look at
2nd Line of Defence
(Model Validation & Other)
Enterprise Governance
Model Methodology/Performance
Data & IT Infrastructure
Process & Controls
Source: Ernst & Young
methodology review and process review
within this framework.
Model risk management
framework
Traditionally, model risk management often
focused solely on the independent model
validation function. Increasingly, however,
institutions are taking a more inclusive
approach to identify sources of model risk
and to ensure that adequate controls are in
place over the entire model life cycle.
The traditional three lines of defence can
be summarised as follows in the context of
model risk:
First in line are the model-owners –
typically models are owned by various units
and functions within the business. They are
responsible for managing the models over
their life cycle. Notice that this model
27
TMISA12 Ernst & Young2_Layout 1 15/11/2012 12:19 Page 28

Managing Treasury in the New Economy

Figure 2 of what is done, but also how it is done. A

Fig 2 process review should focus on roles and
responsibilities, compliance with policies and
Model Review Components
Model
procedures and reviewing the effectiveness
Process & Data & IT
Governance Methodology Controls
of controls. The standard processes involved
Infrastructure
/ Performance
in a model’s life cycle are characterised by
Business Purpose PR MR
the following steps:

Model Development PR MR PR PR Business Purpose Stage

What will the model be used for and what
Implementation PR MR PR PR
Model
Model will it not be used for? (The two questions
Model Operation
are equally important.)
Lifecycle
Lifecycle PR MR PR PR

Ongoing Performance Development Stage

Monitoring PR MR PR
This stage involves several processes,
including selecting methodology, collecting
Change Management PR MR PR PR
data, involving internal or external experts,
Legend: (MR) Methodology Review (PR) Process Review statistical analysis, and other contributing
factors in the developmental stages.
Source: Ernst & Young
Implementation Stage
During this stage IT systems are changed to
make the model run smoothly. Testing and
communication processes are also
implemented. Which tests were conducted
after implementation? How were current
Successful model governance requires not only systems changed to accommodate the
consideration of what is done, but also how it is done. model? Was the necessary process followed
to communicate with those areas of the
business that control the inputs to the
model, as well as those areas of the business
that use the outputs of the model?

Model Operation Stage

ownership does not preclude that many of the
actual modelling/pricing tasks are delegated
to centres of excellence that sit, for example,
in a risk control cell.
The second line of defence includes
enterprise governance and the four different
model review components. Committee and
functional responsibilities for enterprise
governance of models typically include the
following four components: (1) definition of
models, (2) maintenance of a model
inventory, (3) maintenance of policies and
guidelines, and (4) required model processes
and controls.
The responsibilities for the model review
components, for assessing individual models
and enterprise model governance, are typically
the responsibility of an independent validation
function and other control functions.
Finally, internal audit oversees all aspects
of the model risk management framework
and has a key role in covering model review
components, including those that are not the
focus of the independent validation function.
Figure one shows a summary of this model
life cycle, including the other two lines of
defence.
Within an effective model risk
management framework, model reviews form
an integral part of the assessment of risk. This
model review process can be split into two
sections, namely process review and
methodology review. Each of the model life
cycle components is considered under the
model review structure, either with or
without the inclusion of a process or
methodology review. These are discussed in
the next two sections.
Figure 2 provides a graphical representa-
tion of this split.
Process review
A process review forms a key part of the
governance of a model. Successful model
governance requires not only consideration
This stage involves processes for maintaining
the model, processes for over-riding model
output, and determining who gains access to
operate the model, and similar processes
involving user input.
On-going Performance Monitoring Stage
Models should be monitored on a regular
basis to make sure that output remains fit
for purpose and that input data is still
appropriate. A process for monitoring the
model should be clearly stipulated, including
measures for regular monitoring and deter-
mination of trigger levels.
Change Management Stage
This stage involves processes relating to
changing the model. These changes could
include anything from slight enhancements
or recalibrations to a total reconstruction of
the model, depending on the recommenda-
tions derived from the performance
monitoring stage.

28 TMI | Special report

TMISA12 Ernst & Young2_Layout 1 15/11/2012 12:19 Page 29
Managing Treasury in the New Economy
Each of these stages consists of one or more
processes which should ensure the
appropriate form and degree of governance.
These processes are designed to govern
activities and thereby minimise the
numerous risks associated with developing
and using models. A process review would
also consider whether the appropriate docu-
mentation exists for each of the steps. As
such, compliance to these processes should
be reviewed on a regular basis.
Methodology review
Model methodology refers to the core
assumptions and calculations performed by
the model. The model methodology is often
thought of as a black box that produces
answers based on the parameters and data
fed into the model. This part of the model can
be extremely complex, but not necessarily so.
It all depends on the purpose of the model
and the type of inputs that are available.
A model methodology review will
typically consist of a series of activities
which can be divided into four steps. The
first step is to ensure that all the
assumptions related to the methodology are
appropriate. This includes assumptions about
data, parameters, statistical distributions,
and other assumptions that might influence
the model.
The next step would be to revisit the
theoretical underpinnings of the model.
Start with basic principles: which type of
calculation does the theory cater for? Are
the necessary conditions for using this
theory met? What are the weaknesses of the
theory and how are these accounted for in
the model. A very important part of the
methodology review is to ensure that the
methodology that is applied is still
FTI STAR - The Treasury Management System
Designed by Treasurers, for Treasurers
Contact: EDDIE FOGARTY efogarty@ftitreasury.com
TMI | Special report
appropriate. This should be done before rather fully understood and put to its
checking whether the methodology is intended use.
correctly applied.
Correct application is the next considera-
tion – the third step is to ensure that the
Conclusion
theory (formula or methodology) is correctly The effective management of model risk is
applied in the model. Models are often an increasingly important consideration in
implemented in the form of computer reducing the risk associated with it. This
algorithms. It would therefore be necessary to article provided a brief introduction to the
check whether the coding of the algorithm is concept of model risk and an associated
correct and whether it executes correctly. The model risk framework which can be used to
final step is to check whether the data and minimise this risk significantly. The imple-
other input parameters are appropriate for mentation of a full model risk review
the model or whether changes, such as trans- includes the use of process and methodology
formations, are required. reviews as a thorough and proactive model
Following these steps will ensure that the governance strategy for assessing and
model, especially the model methodology, is mitigating model risk within the broader
not treated like a mysterious black box, but model risk management framework. ■
Carel van der Merwe
Quantitative Advisory Consultant
Carel.VanDerMerwe@za.ey.com
Carel joined Ernst & Young after completing his master's degree in
Financial Risk Management at Stellenbosch University. While doing
his master’s he lectured on various subjects including Statistics and
Financial Risk Management. At Ernst & Young he has performed extensive IAS39 and
IFRS2 valuations for a variety of audit clients. Carel has also acted as a quantitative
consultant on various other advisory projects.
Jaco van Tonder
Quantitative Advisory Consultant
Jaco.VanTonder@za.ey.com
Jaco is currently completing his master’s in Development Finance at
the USB. He has had a more focused role within the credit risk space
where he performs detailed analytical reviews of various components
of the impairment calculations. In some cases re-performances are required for key
impairment components as well. Jaco also forms part of the SAICA/ASSA Model
Matters working groups for Retail Credit, Operational Risk, and Corporate Credit.
www.ftitreasury.com
29