Tripod Beta and Performance Audit

Paper to be presented at the International Seminar on Performance Auditing,

Oslo, 23-25 May 2007

*** Preliminary draft ***

Version of 3 May 2007

dr Rudi Turksema
dr Koos Postma
Alice de Haan MA
Netherlands Court of Audit
PO Box 20015
2500 EA The Hague
The Netherlands
R.Turksema@rekenkamer.nl
www.rekenkamer.nl
1. Introduction

Performance audits at the Netherlands Court of Audit (NCA) focus on ‘the relationship between

policy and implementation’ (NCA, 2003). This topic involves examining the gap that we regularly

uncover between (a) the aims of policy and the government’s aspirations that it embodies and (b)

how these are achieved in terms of actual performance and social effects. Next to that, we try to

explain the gap: why has policy not been implemented as planned, why are targets not being

achieved? Good explanations enable us to make better (more accurate) recommendations.

In our performance audits we frequently use case studies, interviews and literature survey.

Sometimes we use more advanced quantitative methods, such as panel regression models. The

international peer review of our audit office (SAI’s, 2007) concluded that we sometimes collect too

much information. This leads to very detailed reports of findings and makes it difficult to reach the

necessary conclusions and recommendations. Therefore the peer review team recommended us to

expand our range of methodologies.

This paper describes one of the methods that we have recently added to our methodological toolkit,

Tripod Beta. This method is very useful to enhance the explanatory power of qualitative data. It

may lead to findings that are ‘analytically representative’, i.d. same processes will take place under

similar circumstances (cf Yin, 2003).

What the method Tripod Beta entails is described in section 2. In section 3 we present the

application of Tripod Beta in a performance audit on the protection (conservation) of nature areas.

Section 4 offers a summary and conclusion.

2. Tripod Beta

2.1 What is Tripod Beta? 1

Tripod Beta is an instrument to conduct an analysis of an incident, e.g. an explosion in a fuel plant. The

Tripod theory originated from research undertaken in the late 1980s and early 1990s into the

contribution of human behavioral factors in accidents. The research, by the Leiden University and the

Victoria University, Manchester, was commissioned by Shell International. Originally, Tripod Beta was

mainly used in the petrochemical industry. Later on, it was also applied in other fields such as incidents

in jails, a fire in a refugee camp, etcetera. In this paper we will demonstrate that the method is very

suitable for finding explanations for a gap between policy and implementation as well.

1
Information on Tripod Beta from: Tripod Beta. Incident Analysis Primer. Source: www.tripodsolutions.net.

1
Incidents occur when actors make a wrong decision. But the focus of Tripod Beta incident analysis is on

possible latent failures in an organization that ‘invite’ these actors to fail. The objective of an incident

investigation and analysis is to identify and correct the immediate and underlying causes that created, or

contributed to, an incident and so prevent its future recurrence.

2.2 Incident analysis and causes

Incidents occur when barriers fail to prevent actors to make decisions that may escalate to undesirable

consequences. The barriers can be of different types e.g. related to design, systems, procedures,

equipment etcetera. The barriers are put in place and kept in place by people with the competence to do

so, in line with standards and specifications. Incidents happen when people make errors and fail to keep

the barriers functional or in place e.g. people doing the wrong thing or people not doing what they should

do.

Incident analysis

Incident analysis consists of three steps:

• to identify the chain of events from the cause of harm to the outcome; the undesirable

consequences;

• to identify the barriers that should have stopped the chain of events;

• to identify the reason for failure of each of the barriers.

Figure 1 illustrates the three steps in an incident analysis.

Figure 1. Failing barriers make undesirable consequences possible.

Source: Tripod Beta Foundation (2006).

2
Incident causes

One of the most important aspects of Tripod Beta is that it assumes that human behavior resulting in

failures is influenced (determined) by latent failures in the context of an organization. If these latent

failures are not addressed, only symptoms are being tackled. This is why a human behavior theory is a

key element of Tripod Beta. According to this theory human error will always exist. It is therefore

essential to make sure that there are adequate barriers in place. Figure 2 illustrates this.

Figure 2. Underlying causes, preconditions and immediate causes.

Source: Tripod Beta Foundation (2006).

The immediate cause of incident is the behavior of humans. Mistakes or violations are types of behavior

that may lead to incidents. They are the result of ill-conceived intentions. Together with individual’s

perceptions and belief, these intentions are preconditions that enable that an incident can happen. These

preconditions, in turn, are affected by the underlying causes. These underlying causes can be found in

the influencing environment, e.g. the context of an organization. They are the primary object of analysis

as they can help us understand how people develop intentions or plan and how this ultimately affects

their behavior.

The underlying causes can be divided into ten basic risk factors (types of latent failures). They are

related with:

• Hardware • Housekeeping

• Design • Incompatible goals

3
 • Maintenance Management • Communication

• Procedures • Organization

• Error-enforcing conditions • Training

These basic risk factors play a central role in the data collection. The questionnaires that are used in the

interviews are elaborated on these basic risk factors. Also other relevant information from other sources

can be added to the data (information from dossiers etcetera).

2.3 Tripod Tree

With the Tripod Beta software, auditors can make a Tripod Tree which makes explicit which failed

barriers contribute to a particular incident. Central elements in such a tree are the agent of a change, the

object changed and the resulting incident event. These describe what happened. It also shows the

barriers that should have stopped the incident, i.e. how it happened (the immediate cause). Next to

that, preconditions and underlying causes can be included in a Tripod Tree.

Figure 3. Basic Tripod Beta Tree.

Source: Tripod Beta Foundation (2006).

Iterative process

Constructing such a Tripod Tree is in itself an iterative investigation process. Preliminary versions of the

Tripod Tree are further investigated which leads to validation and refinement of the Tripod Tree. This

iterative process stops when all relevant facts have been identified and the Tripod Beta tree accurately

reflects the incident.

4
3. Tripod Beta and Performance Audit

Tripod Beta has been used in an audit of government’s success in finding ways to compensate for the

loss of nature. The ‘incident’ was: not complying with laws and regulations. Using Tripod Beta, we have

identified the underlying latent errors and the context that makes that people make an active error (not

complying). The latent error is seen as an explanation as to why policy has not been implemented or as

to why it did not have the intended effect.

3.1 The audit problem

As in many European countries, the quality of nature in Netherlands is deteriorating. This is due to many

reasons amongst which those related with spatial planning, such as laying out of roads, housing estates,

and industrial sites. Whenever a new road or housing estate is build, nature will be affected, in terms of

size as well as in terms of coherence. Moreover, as a result of damage to the environment, the quality of

nature has been worsened.

Size of nature

Figure 4 below, illustrates the change in the size of nature between 1990 and 2000. The rapid decline is

caused by changes in the way scarce space is being used. In the beginning many nature areas were

reclaimed for agricultural purposes. Later on, intensified use of infrastructure, the growth of recreational

use and building more houses, have affected the size of nature.

Figure 4. Size of nature in the Netherlands between 1900 and 2000.

Source: Statistics Netherlands Statline (accessed 2006) .

5
Coherence of nature

Increased building activities and intensified use of infrastructure have not only affected the size of

nature, but also the coherence of nature. Scientists believe large, continuous areas that are linked

together are a precondition for the protection of nature. The distance between suitable habitats, the size

of such a habitat affects the immigration possibilities and the risk of extinction. Next to that, there is less

chance of disturbance by humans in larger, coherent areas.

Quality of nature

The quality of nature in nature areas is deteriorating. This is a result of the deterioration of the quality of

the environment, indicated by nitrogen deposition, acidification of the soil, and groundwater depletion.

These conditions are mainly affected by human behavior. Intensive agriculture leads to nitrogen

deposition en groundwater depletion. Building, industry, agriculture, and the intensified infrastructure

cause increased air pollution (acidification). These environmental problems cause further deterioration of

the quality of nearby nature areas.

3.2 Government policy: spatial protection

There is of course government policy that tries to address these problems. Given the collective good

character of nature, central government has an important responsibility for the protection and expansion

of nature in the Netherlands. Government policy can be described as follows:

Areas Species

Expansion of nature values Nature expansion Active species policy

Protection of species

Protection of existing nature values Nature area protection (or passive

species policy)

Source: Kuindersma et al (2004).

This audit focuses on the protection of existing nature areas. The expansion of nature has been the

subject of our audit into the achievements of the National Ecological Network (NCA, 2006). In that audit,

the progress of the realization of nature areas in size, coherence, and quality is described. Also,

explanations are given for problems in policy implementation.

Existing nature areas are protected by two principles: ‘no unless’ and ‘nature compensation’. Under both

Dutch and European legislation, the government may encroach upon protected nature areas only if there

6
are important social reasons for doing so and it must provide compensation in the form of new nature

areas. In these protected nature areas, there is an obligation to conserve the essential features and

values and implement a “no, unless” regime’. The basic protection is designed to prevent irreversible

interventions in relation to the future function’ (Ministries of VROM, LNV, V&W and EZ, 2006, p. 113).

No, unless → In nature areas where the “no, unless” regime’ is in effect, no new plans, projects, or

actions are allowed if they seriously affect crucial characteristics and values of the area, unless there are

no real alternatives and if there are important social reasons for doing so.

Nature compensation → If this is the case, damage to nature has to be minimized by mitigating

measures as much as possible. Residual damage has to be compensated.

3.3 Actors involved

Several actors are involved in this policy. The ministries of LNV (Agriculture) and VROM (Spatial

Planning) are responsible for legislation with respect to nature protection and spatial planning. Provinces

and municipalities implement spatial policy (and therefore nature policy). Provinces indicate where the

boundaries of nature areas are, indicate the types of nature goals, and do the acquisition of land.

Municipalities, but also private parties, suggest projects that may do damage to protected nature areas.

3.4 Research problem

Several sources indicate that there are problems with the implementation of the nature protection policy

(e.g. Werkgroep Vogel- en Habitatrichtlijnen (2003); Gijsen (2003); VROM-Inspectie (2006)). This

means that in some cases it is insufficiently demonstrated what the social reason is for damaging nature,

that in some cases alternatives are not sufficiently considered, and that mitigating measures are not

applied. Next to that, the effectiveness of actual compensation for damage to nature is unclear, as the

quality of this compensation will only be known in the future.

Therefore our research problem is:

• Does the implementation of nature compensation measures contribute to nature protection and

by that to a sustainable use of space?

• Does government succeed in an effective and efficient implementation of nature compensation

policy, e.g. when industrial sites, roads, housing estates are constructed?

• If not, how can that be explained?

7
3.5 Research design

Several methods, such as desk research, interviews, and case studies have been used to collect

data. Desk research and interviews were mainly used to describe the changes in nature and to get

a good understanding of nature protection policy. Case studies, in combination with Tripod Beta,

were used for fact finding and findings explanations.

Case studies

Many interventions that affect nature and where the “no, unless” regime is in effect, occur at the level

of the municipality. Therefore it is necessary that we focus our main attention at the local level. If we do

not descend to this micro level, it will be difficult to find plausible explanations for ineffective policy.

The selection of cases was not easy, as there is no nationwide monitoring of interventions where

the “no, unless” regime might apply and where nature has to be compensated. So, we selected ten

interventions in provinces where there relatively much protected nature (Overijssel, Gelderland, Noord-

Brabant and Limburg). For these interventions we were able to use information from the Inspectorate

responsible for supervision on spatial planning.

Topics

We checked whether the following steps had been taken whenever there was an intervention where

the “no, unless” regime applied:

• Identify how the intervention affects nature;

• Whether sufficient attention been paid to alternative solutions;

• Whether important social reasons for the intervention are demonstrated, e.g. prevention of

flooding, employment, better accessibility;

• Minimizing damage to nature through mitigating measures, when there are no alternatives;

• Compensation of the residual damage to nature, such that there is no deterioration of

nature, in terms of size, coherence and quality.

Data collection

Data have been collected through:

• Dossier analysis;

• Interviews with those involved: civil servants of the municipalities, Aldermen, civil servants

of the provinces, member of the Provincial Executive, representatives of nature

organizations, private promoters. In the interviews we have used an extensive

questionnaire that has been developed on the basis of Tripod Beta.

8
3.6 Applying Tripod Beat: from incident analysis to policy analysis

As said before, the ‘incident’ in performance audit was: not complying with laws and regulations. In our

analysis we have identified several bottle-necks in the implementation of nature protection policy.

Figure 5 shows the Tripod Tree that we have constructed in our audit. The red objects describe the laws

and regulations. Whenever there were broken barriers, the underlying causes and preconditions have

been identified.

Vanafhetbeginvan
hetprojectwordt
compensatieingepland.

Bijinitiatiefnem
er
weinigkennisvan
regelgeving
Initiatiefnemergebruikt
Gem eentegeeftgeen Initiatiefnemerheeft denieuwaangelegde erisnietsgeregeld
voorlichtingovereisen geenbelangbijextra natuurvooreigen voorhetbeheervande
vannee,tenzijbeginsel werkzaamheden bedrijf(inditgevaleen natuurcompensatie
recreatiepark)

nauwelijksonderzoek Natuurcompensatie
Particuliere
uitgevoerdnaar Gemeentesteltgeen wordtgebruiktals
initiatiefnem erverzuim
t
alternatievenen eisenencontroleert recreatiegebied
eencompensatieplan
ecologischeeffecten. niet Gemeentevoertgeen (picknickplaatseen
Gemeentemaakt optestelen Aantalha
initiatiefnemers Weinigonderbouwing… Inititiefnemerheeft controleuitopde natuurcompensatieis zwemvijver)
weinigbetrokkenheid kwaliteitvande
onvoldoendeduidelijk geenkennisvanwet- duidelijk.Kwaliteit
vanuitdegemeente
welkeonderzoekenze enregelgeving natuurcompensatie com pensatieisniet
uitmoetenvoeren duidelijk. Geeneisen…

com penseren(voordat
aanvraagvergunning/ vaststelen Uitvoeringen
verwerkenvergunning toetsgemeente nee,tenzijbeginsel Compensatiebeginsel Compensatieplan beoordelingprovincie bezwaarenberoep jenatuurbeschadigd
projectidee bestemmingsplan handhaving
hebt)
gemeenteisontevreden
Nee,tenzijwordt Geenmintigerende Geencompensatieplan Gemeenteheeftgeen
overdekwaliteitvande
nauwelijksuitgevoerd maatregelenbekeken opgesteld handhavingsinsrumenten
natuurcompensatie

Vanafhetbeginvan Nalevingvan
hetprojectwordt regelgevingis
compensatieingepland probleem
Binneneenproject Initatiefnemeris
gaatenmeteenover waarschijnlijkaleen
richtinghetplannenen ingelichtoverde
uitvoerenvan verplichtingtot
compensatieverplichti… com penseren.

Figure 5. Tripod Tree in this performance audit.

3.7 Results

The overall conclusion of the audit is that nature protection policy is not effective. Our analyses indicate

that this is mainly a result of policy implementation in stead of policy theory. Both the “no, unless”

regime and the nature compensation regime are applied insufficiently. They provide no effective

protection of nature. At best, they slow down the actual intervention that causes damage to nature. If

nature is being compensated (which was only the case in fifty percent of our cases), the quality of ‘new

nature’ is not the same as the quality of ‘old nature’.

Underlying causes are found in the context of the policy actors, such as the way policy implementation is

organized, communication, and conflicting goals or interests. Some of the relevant actors did not have all

of the information necessary to correctly apply the “no, unless” regime. Sometimes the information was

9
known, but by a colleague from a different department, who sometimes had different interests. Another

problem that we identified, was that there is no clear description of what ‘damage to nature’ actually is.

How bad should it deteriorate to prevent the intervention from happening? The same goes for the

importance of social problem at hand: what is important and who says so?

These problems in the implementation of nature protection policy make that government policy is not

effective: protected nature will be affected, in terms of size, coherence, and quality.

4. Summary and conclusion

In this paper we described the application of the method Tripod Beta in a performance audit on the

protection of nature areas.

We think Tripod Beta is a welcome addition to the methodological toolkit of performance audits. It

can help audit teams structure their audit and find answers to the ‘why’ question, because it deals

systematically with the analysis of the reasons for failure of policy (or barriers in general). This

also enables the audit team to formulate more accurate recommendations, which may improve the

effectiveness of our audit work and the effectiveness of government policy.

References

Gijsen, J.J.C., R.I. van Dam R.I., and A.H. Prins (2003). Natuurcompensatie. Hoe werkt het in de

praktijk? Werkdocument 2003/13. Wageningen: Alterra, Research instituut voor de Groene

Ruimte.

Kuindersma, W., F.H. Kistekas, and R.C. Apeldoorn (2004). De transformatie van Nederlands

natuurbeleid door Europees recht. Een analyse van de gevolgen van de implementatie van

de Vogel- en Habitatrichtlijn [The transformation of Dutch nature policy as a result of

European law. An analysis of the consequences of the implementation of the Birds and

Habitats Directive]. Wageningen: Planbureaustudie nr. 8 Natuurplanbureau.

Ministries of VROM, LNV, V&W & EZ (2006). National Spatial Strategy, Part 4. The Hague: Sdu.

Netherlands Court of Audit (2003). Performance and operation of public administration, Strategy

2004-2009. The Hague: Netherlands Court of Audit.

Netherlands Court of Audit (2006). National Ecological Network. The Hague: Netherlands Court of

Audit.

Netherlands Court of Audit (forthcoming). Bescherming van natuurgebieden [Protection of nature

areas]. The Hague: Netherlands Court of Audit.

10
SAI’s of Norway, New Zealand, South Africa and the UK (2007). Peer review of the Netherlands

Court of Audit.

Tripod Beta Foundation (2006). Incident Analysis Primer. Source: www.tripodsolutions.net.

VROM-inspectie (2006). De uitvoering van het compensatiebeginsel. Bij ruimtelijke ingrepen in de

Ecologische Hoofdstructuur. Arnhem/Eindhoven.

Werkgroep Vogel- en Habitatrichtlijnen (2003). Nederland op slot? De 98 Europese en Nederlandse

natuurbeschermingswetgeving nader bezien. Interdepartementaal Beleidsonderzoek 2002-

2003, nr. 7. Bijlage bij Tweede Kamerstuk 2002-2003, 29 043 nr. 1.

Yin, R.K. (2003). Case Study Research. Thousand Oaks, CA: Sage Publications.

11