12/15/2019 Module 4.

Planning an audit on water-related issues

Module 4. Planning an audit on water-

related issues
This module discusses the audit design phase with a focus on choosing the focus of the audit
and developing the Audit Design Matrix. Practical examples are provided on performance audits
on water-related issues, and cooperative water audits are also treated.

By completing the module, one should be able to undertake a preliminary study to devise a
water-related audit topic relevant to their country; read and analyse performance audit design
matrices addressing water-related issues as well as to independently draft an audit design
matrix for a water audit.

Powered by Panopto

Video 4. Planning an audit on water-related issues

4.1. Selecting audit focus

For general information on planning a performance audit, please visit MOOC: Introduction on
Environmental Auditing. In this module, emphasis is put on practical examples and
recommendations connected specifically to water audits.

In previous modules you have observed the variety and complexity of issues that are all
connected to water and water management. Upon starting to plan an audit on water issues, as
in any performance audit, the following steps need to be undertaken: selection of a topic,
preliminary study and audit design. The exact format and sequence (or overlap) of these
procedures may vary in different audit offices.

The first step in planning an audit is selecting the audit topic. The general topic selection
process is often connected to the regular monitoring activities of auditors and in-depth
preliminary study will already be undertaken to determine the exact focus of the audit.

It is necessary to know and understand the main water issues in your country. What are the
biggest problems? What are the current trends? Maybe the population is increasing in major
towns and there is a need for new infrastructure. Or maybe, increasingly, houses are

https://sisu.ut.ee/water/book/export/html/19772 1/10
12/15/2019 Module 4. Planning an audit on water-related issues

constructed on the shoreline and there is a considerable risk of floods affecting the environment,
people and their property. There may also be a risk of conflicting policy goals (e.g. agriculture
versus protection of groundwater), poor enforcement (e.g. scarce groundwater resources are
poorly managed), accidents (spills) or something else.

THINKING EXERCISE
Recall Module 1 of this course. What are the biggest water related problems in your
country?

Has drinking water quality improved over time? Has the status of water bodies
improved?

Auditor should also find information about the relations between water management/protection
and the national budget of the government in order to determine the relevance of a given topic
by monetary value. Often this information is not easily available as resources needed to secure
different water management and protection functions may be under different budget allocations,
externally financed, privatized through concessions and leases etc. The auditor should in this
case also determine which institutions (e.g. department, ministry) are responsible for different
areas of water management and what proportion of their budget is allocated for these functions;
and additionally, consider the cost for citizens.

There are several other stakeholders and interest groups besides state institutions, such as
consumer organisations, health organisations, companies (e.g. producers of food and
beverages), scientists, NGOs. For mapping the problems, auditor may read their publications,
participate in seminars and organize face-to-face meetings.

Methods such as familiarising oneself with relevant legislation, key data, studies and articles in
the media as well as conducting several interviews and perhaps focus group discussions, help
an auditor to achieve a comprehensive overview of problems and risks as well as their root
causes. A thorough preliminary study helps to avoid wasting time at later stages as the audit is
better focused.

Tip for auditors

Aspects to consider during the preliminary study of a water audit (recall all previous
Modules of this course):

Have standards for drinking water quality been established in your country? What
about the status of water bodies and groundwater?
Is monitoring data available about the quality of drinking water? What about the
status of water bodies? Do the monitoring results indicate any shortcomings? What
are the trends?
Is the legislative framework for water protection and management in place in your
country? What requirements and goals are established therein? Are the regulations
being implemented?
Which international and regional agreements has your country signed and what
responsibilities ensue from these?
What about strategic water management documents and goals established therein?
Have the targets been achieved?
What economic instruments are applied for water management in your country? Do
they work as intended, e.g. cover all the costs, create incentives for behavioural
change or create revenue for environmental investments?
Which institutions are responsible for water management? Are private companies
involved? Are the responsibilities of different stakeholders clear and logical?

https://sisu.ut.ee/water/book/export/html/19772 2/10
12/15/2019 Module 4. Planning an audit on water-related issues

It is recommended to focus on a comprehensible topic. An attempt to cover everything within

one audit creates a risk of losing focus and wasting time on trying to determine all possible
connections. Focusing an audit is also necessary to avoid wasting resources on studying issues
which will not be reflected in the audit report. Table 3 provides some examples of possible
focuses of a water audit.

Table 3: Examples of both wide and focused audit topics

Wide topic Focused audit topics

Water quality in rivers and lakes Effectiveness of preventing pollution from diffuse sources

Effectiveness of preventing pollution from farms

Effectiveness of preventing pollution from households

Appropriateness of river basin management plans

Implementation of river basin management plans

Drinking and/or wastewater Quality (including safety) of drinking water

Efficiency of water infrastructure projects and management

Price of water management

Groundwater Effectiveness of managing scarce water resources

Protection of groundwater from diffuse pollution

Floods Readiness of managing flood risk

Appropriateness of flood management plans

Using the INTOSAI water guidelines, a number of SAIs have audited various water topics.
These include:

The availability of safe drinking water, e.g. whether the drinking water supplies have been
effectively managed to meet the likely future demand; the ensuring of access to drinking
water; the sustainability of the price of drinking water.
Competing demands for limited water supplies, e.g. potential impacts of mining activities
on water supplies; management of competing demands between supplying drinking water,
agricultural and industrial needs and ecological concerns.
Drought preparedness and management, e.g. management of water shortage during
drought; effectiveness of government efforts to mitigate the consequences of droughts.
Flood management, e.g. performance of flood prevention and protection measures; use of
funds for flood prevention and response.
The quality of surface waters, e.g. effectiveness of efforts to protect a given water body
from pollution; examination and assessment of the sources of nutrient pollution.
The marine environment (often cooperative audits), e.g. management of the impacts of
pollution on marine environment; preparedness to compact pollution from ships.
Planning and financing for water infrastructure, e.g. planning, financing and maintenance
of water infrastructure projects, including in view of access to drinking water and
sanitation.
https://sisu.ut.ee/water/book/export/html/19772 3/10
12/15/2019 Module 4. Planning an audit on water-related issues

Implementation and enforcement of water laws.

The challenges of managing water resources shared by multiple nations.
The adequacy of water-related data; e.g. in the framework of the effectiveness of the
management of water resources.
The impacts of climate change on water resources; e.g. effectiveness of efforts to address
or adapt to the water-related impacts of climate change (floods).

After the preliminary study, the auditor is ready to decide the focus of the audit and formulate an
audit objective. Below are two examples of audit objectives which actually state the focus of an
audit.

Audit objective A: Evaluate the effectiveness of protecting groundwater from diffuse

pollution from agriculture
Audit objective B: Evaluate the efficiency of management of water infrastructure in
municipality X

4.2. Audit design matrices for selected water issues

When the audit objective is decided, it is time to start planning the audit. Deriving from audit
focus, auditors have to phrase the audit objective (e.g. evaluate the effectiveness of protecting
groundwater from diffuse pollution from agriculture).

Next, one needs to think about audit questions which need to be answered in order to meet
the audit objective. Audit questions are usually broken into sub-questions. It is common to use
closed (yes/no) questions in audit. Sub-questions are selected in such way that, if they are
answered positively, the main question is also answered positively.

If necessary, open questions (why?) are added to better understand the reasons for problems
under scrutiny.

For evaluating the situation, each audit question is equipped with criteria - often deriving from
legal acts, but also strategic documents and international agreements. Due to its rather detailed
nature, the Water Framework Directive serves as a good source of criteria for the audit offices of
the EU countries.

https://sisu.ut.ee/water/book/export/html/19772 4/10
12/15/2019 Module 4. Planning an audit on water-related issues

Criteria describe the desired situation and are the main basis for the auditor’s evaluation. In
case of absence of legal acts, auditors may use criteria which have been discussed and agreed
upon with the auditee.

At this point, auditors need to consider which methods are most suitable for collecting and
analysing information. Methods used in audits related to water management do not differ
substantially from other performance audits. However, in case of a lack of data on water quality
or the questionable quality of such data, external experts may be hired.

Audit design matrix (ADM) is a common tool widely used by auditors for organising audit
questions, criteria and selecting suitable audit methods. Below you can see two audit matrices.

For more information on designing an ADM please visit MOOC: Introduction on Environmental
Auditing.

Table 4: Example of an audit matrix

Audit title: Protection of groundwater from agriculture

Audit objective: Evaluate the effectiveness of protecting groundwater from diffuse pollution
from agriculture

Audit question 1: Has the quality of groundwater improved over time?

Sub-questions Methods Expected findings

(incl.
Criteria
information
sources)

1. Is the quality
of groundwater National monitoring Review of Water quality is monitored only
monitored programme includes monitoring in Z region.
regularly? monitoring of data
groundwater in C %
of wells at least in D
years.

2. Is the quality Water quality Analysis of Water quality does not meet
of water corresponds to the monitoring quality standards in all wells.
maintained? data Deterioration of water quality
https://sisu.ut.ee/water/book/export/html/19772 5/10
12/15/2019 Module 4. Planning an audit on water-related issues

indicators of good has been noticed in Y% of

water status wells.

Audit question 2: Has the government established requirements for preventing diffuse
pollution?

1. Does the
water law The water law is Legal Water law regulates only
regulate diffuse harmonised to WFD; analysis pollution from large
pollution? water law includes municipalities not from small
provisions for households.
preventing pollution
from all households

2. Does the
RBMP include Targets for reducing Legal Targets for major sources of
targets for diffuse pollution as analysis diffuse pollution are outlined in
reducing diffuse set in the RBMP the RBMP, but deadlines are
pollution? very long (30 years).

3. Are there
regulations for Legislation includes Legal There is a good practice
limiting the use provisions for the analysis guideline for farmers.
of fertilisers and use of pesticides
pesticides? and fertilisers
according to the
WFD

Audit question 3: Are the regulations for protecting groundwater enforced?

1. Are the main

polluters known Pollution sources Review of the Some pollution sources that
to the are analysed in an RBMP and its were recently discussed in the
government? inventory to prepare planning media were not known to the
the RBMP; documents; environmental inspectorate
The environmental until its members got such
inspectorate is Interview with information from newspaper
monitoring the officials sources.
pollution levels responsible
regularly and finds for the RBMP;
the pollution
sources. interview in
the
environmental
inspectorate

2. Do farmers
follow the good 90% of farmers Online Some farmers follow the
practice follow the good survey; guideline; practice is varied.
guideline? practice guideline
Interviews
with 10
farmers
(selected
based on the
online
survey).

https://sisu.ut.ee/water/book/export/html/19772 6/10
12/15/2019 Module 4. Planning an audit on water-related issues

Table 5: Example 2 of an audit matrix

Audit title: Drinking water management

Audit objective: Evaluate the effectiveness of the management of drinking water supply

Main audit question: Is the provision of drinking water supply effectively organized?

Sub-questions Methods (incl. Expected findings

Criteria information
sources)

1. Have legal
provisions and a Systemic legal Analysis of A coherent legal framework is
policy provisions and a relevant missing, and the activities of
framework policy framework international various agencies need to be
related to are in place and and domestic more coordinated; Objectives
drinking water address risks legislation; set in legislation need clear
supply been related to the supply expression through policy and
established? and safety of Document planning instruments.
drinking water; analysis (e.g.
Responsible water
agencies have been management
identified along with plans);
clear roles and
responsibilities, and Interviews,
activities are questionnaires
coordinated

2. Is monitoring
of drinking water Monitoring Analysis of Monitoring arrangements are
supply programmes have relevant in place, but there are
effective? been established legislation and shortcomings in the regularity
and are adequately documentation; and comprehensiveness of
funded; quality testing. Facilities for
Identified Interviews, water quality testing need
shortcomings are questionnaires; investments.
addressed and
eliminated; Data collection
Quality standards (e.g.
for drinking water monitoring
are met data, permits)

3. Is access to
safe drinking All public water Review of X share of the population is
water services supply systems monitoring connected to water supply
ensured? meet quality data on systems in which the drinking
standards; drinking water water does not meet quality
Information on non- supply standards; Corresponding
compliance and systems; data needs to be publicly
potential risks accessible.
connected to the Analysis of
quality of drinking reasons for
non-

https://sisu.ut.ee/water/book/export/html/19772 7/10
12/15/2019 Module 4. Planning an audit on water-related issues

water is publicly compliance

available; (interviews with
The price of drinking owners);
water is not an
obstacle to drinking- Review of
water access publicly
available data

Expert financial
analysis of
price of water

4. Are the
pricing Analysis of Tariffs must be revised as they
principles Service prices of relevant will not cover the costs of
established in a water supply are set legislation and water suppliers in the long
financially after evaluating documentation term and rehabilitation of
sustainable required costs, and on pricing; ageing supply systems may
way? allow for necessary prove difficult without
investments; Interviews; government subsidies.
Continuity of the
operation of water Data collection;
suppliers is ensured
Review of
international
(best) practices

Table 6: Example 3 of an audit matrix

Audit title: Effectiveness of the financial programme X in protecting Y river basin

Audit objective: Evaluate whether the programme X for enhancing the status of water bodies
in poor and bad status in Y river basin has been effective and efficient.

Audit question 1: Has the status of water bodies, which were earlier in bad or poor status
improved since the implementation of the programme X?

Sub-questions Criteria Methods (incl. Expected findings

information
sources)

1.1 Was the Objectives of the Comparison of the Most objectives follow the Y
programme programme X objectives of the Y river basin management
designed derive from the Y river basin plan.Some objectives of Y
according to the river basin management plan river basin management plan
objectives of the management and programme X were omitted; however, they
Y river basin plan should have been included.
management
plan?

1.2 Has the water quality is Analysis of Water quality has somewhat
chemical status regularly monitoring data improved. The level of some
https://sisu.ut.ee/water/book/export/html/19772 8/10
12/15/2019 Module 4. Planning an audit on water-related issues

of the water monitored Interviews with chemicals in some water

bodies water quality has experts bodies has remained above
improved? improved the level of standard
according to the
standards
established for
chemical
substances

1.3 Has the Ecological Analysis of Ecological status has

ecological status characteristics monitoring data somewhat improved in some
of water bodies are regularly water bodies. Endangered
improved? monitored Interviews with species have not returned to
experts water bodies under study.
Ecological status has
improved according to
the standards
established for
biological elements
and physicochemical
characteristics

Audit question 2: Was money used efficiently for managing programme X?

Sub-questions Criteria Methods (incl. Expected findings

information
sources)

2.1 Were the Project selection Review of project Most projects follow the
projects selected criteria reflected documentation objectives of the programme,
according to the the objectives of (programme, but some projects fail to meet
objectives of the the programme project proposals, the objectives.
programme? All project met protocols of
the selection selection
criteria committee
meetings)

2.2 Were the All project Analysis of Most projects were carried
projects activities were relevant out according to the plan.
conducted carried out documentation Some activities were not
according to according to the (project reports, carried out in some projects.
conditions of the plan financial reports)
programme? Money was used
according to Interviews
project plans

Tip for auditors

https://sisu.ut.ee/water/book/export/html/19772 9/10
12/15/2019 Module 4. Planning an audit on water-related issues

You may want to explore EUROSAI’s database of audits for summaries of water-related
audits conducted by European Supreme Audit Offices:
https://www.eurosai.org/en/databases/audits/index.html

Click See search filters in the middle of the page

Type „water“ in the open field under Search for text
Tick „Environment and natural resources“ under Filter by subject
Click Filter at the end of the search box

4.3. Cooperative audits

With an increasing number of audits conducted cooperatively by two or more SAIs each year,
cooperative audit is an established practice. A cooperative audit can take a form of a joint audit
whereby one audit team composed of auditors from different SAIs prepares a single report or a
parallel audit whereby individual teams in each participating SAIs issue separate national
reports. A coordinated audit is either a joint audit with separate audit reports or a parallel audit
with both, a single report and national ones. [Ref 49]

SAIs may wish to engage in a cooperative audit on water for several reasons. Transboundary
management of water resources that cross international borders is important as activities
affecting one part of the water body, for example, will most probably have an effect on the
whole. Audit offices can focus on the activities of their governments and also additionaly
scrutinize the cooperation of governments in the management of the shared water body.
Comparing national practices might provide evidence regarding the need for the harmonization
of practices and an increased cooperation to ensure that the condition of shared water
resources would not deteriorate. Cooperative water audits can also help participating SAIs build
and sustain audit capacity.

8 European SAIs collaborated on a coordinated audit on the protection of the Black Sea against
pollution in 2011 [Ref 50]. While it was concluded in the joint report that countries had taken the
main measures to ensure the prevention of pollution from land-based sources, challenges in
fully implementing the polluter-pays principle persisted. Among other things, shortcomings in
national monitoring programmes and lack of harmonization between these programmes in
different countries, as well as gaps in the availability of relevant data were highlighted. The audit
report is available at Join Report on the Results of the Coordinated Parallel Audit on Protection
of the Black Sea against Pollution.

https://sisu.ut.ee/water/book/export/html/19772 10/10