Professional Documents
Culture Documents
com
Technical University of Milan, CQA, DIIAR, P.za Leonardo da Vinci 32, 20133 Milan, Italy
b
Technical University of Milan, DIIAR, P.za Leonardo da Vinci 32, 20133 Milan, Italy
c
Technical University of Milan, CQA, Department of Energetics, P.za Leonardo da Vinci 32, 20133 Milan, Italy
d
Technical University of Milan, TTO, P.za Leonardo da Vinci 32, 20133 Milan, Italy
Received 2 November 2006; accepted 20 January 2007
Available online 23 March 2007
Abstract
EMS is a tool for managing the interaction between the organization and the environment. The aim of an EMS is to improve the overall
environmental performance of the organization. The performance should be monitored through measurements, and managed by indicators. Indicators are variables that summarize or otherwise simplify relevant information about the state of a complex system. A correct evaluation of
environmental performance arises from the choice of adequate raw data and from the relationships among raw data.
This paper, after a short excursus concerning the rule of indicators in environmental performance evaluation and the mean of uncertainty,
proposes an approach to the study and the evaluation, through indicators and indices, of the environmental aspect wastewater discharges
of a Local Authority who is involved in EMS implementation. Particularly, the critical analysis of one of the indices that has been used is
reported. The role of the uncertainty of measurements has been stressed. The results showed that measurement uncertainty is essential for
an efficient data comparison and for a correct evaluation of environmental performance, which, in turn, is essential to guarantee the effectiveness
of the EMS application.
2007 Elsevier Ltd. All rights reserved.
Keywords: Environmental Management System; Environmental performance; Indicators; ISO 14001; Uncertainty of measurements
1. Introduction
An Environmental Management System (EMS) is a part of
an organizations management system (including all human,
economical and infrastructural assets) which aims to manage
the environmental aspects related to its activities, products
and services. Its main and ultimate scope is to improve the
environmental performance of the organizations. The new
ISO 14001:2004 standard [1] defines the environmental
performance as measurable results of an organizations
management of its environmental aspects. To help the
organizations in the process of performance measurement, International Standard Organization (ISO) has developed the
specific standard ISO 14031:00 [2]. Indicators are the main tools
of this standard, and are defined as the specific expression that
provide information about an organizations environmental
performance. Their main scope is to make measurement of
the environmental performance easier for organizations.
Unfortunately, the measurement of environmental performance remains one of the greatest difficulties for the organizations and for the certification/competent bodies [3e6]. In
particular, a factor which is often neglected is the uncertainty
of measurements related to the indicators and indices. On the
contrary, the uncertainty that affects raw data is a crucial issue,
since an indicator can yield a reliable picture of the environmental aspects or performance only if it is based on good-quality
data [7].
518
organization with an EMS should assess its environmental performance against its environmental policy, objectives, targets
and other environmental performance criteria. In fact, an
EMS gives an organized and coherent scheme to properly
deal with environmental issues in organizations, with the
main purpose to improve their environmental performances.
Many authors refer environmental performance and problems related to its evaluation (e.g. [14e19]). Therefore, in
1999, ISO published the standard ISO 14031 that gives guidance on the design and use of environmental performance
evaluation within an organization. However, this issue is still
matter for discussion, as reported in many papers [4e6,20,21].
Several definitions exist for the expression environmental
performance, e.g.:
measurable results of an organizations management of its
environmental aspects (results can be measured against the
organizations environmental policy, environmental objectives, environmental targets and other environmental performance requirements [1]); and
results of an organizations management of its environmental aspects (results may be measured against the organizations environmental policy, objectives and targets)
[2,9].
In any case, in order to evaluate the environmental performance it is necessary to assess the environmental aspects
(element of an organizations activities or products or services
that can interact with the environment [1]). Changes to the environment, either adverse or beneficial, resulting wholly or
partially from environmental aspects, are defined as environmental impacts. The relationship between environmental
aspects and impacts is one of cause and effect.
Besides, a significant environmental aspect is an environmental aspect that has or can have a significant environmental
impact [1]. The organization shall ensure that the significant
environmental aspects are taken into account in establishing,
implementing and maintaining its EMS: identifying significant
environmental aspects and associated impacts is necessary in
order to determine whether and where control or improvement
is needed and to set priorities for management action. In particular, it is necessary to define some significance criteria, that
shall be comprehensive, suitable for independent checking, reproducible and verifiable, in order to identify the significant
environmental aspects of the organizations activities, products
and services. The role of the significant aspects and the related
problems are illustrated in the ISO guidelines (ISO 14004:04
[22]) and those of EMAS Regulation (Recommendation 680/
2001 [23] and Recommendation 532/2003 [24]) and by
some authors (e.g. [3,25e30]).
Usually, to assist organizations in the management of their
environmental significant aspects and impacts it is necessary
to use a tool such as an environmental indicator. The rule of
environmental indicators in the environmental performance
evaluation is essential for many authors (e.g. [16,31e35]). Indicators will support organizations in quantifying and reporting their environmental performances: in fact, it is necessary
to associate one or more indicators to each environmental aspect. Particularly, indicators allow to classify and summarize
data concerning environmental aspects, returning an immediate and representative picture of the company situation in relation to its environmental situation, comparable with the
territorial context of the organization and with the objectives
that have been stated. Indicators should address those environmental impacts that are most significant and which the company can directly influence by its operations, management,
activities, products and services. They should also be sensitive
enough to reflect significant changes in environmental impacts. These values are essential, because they represent the
term of reference for all future environmental performance
evaluations.
4. Indicators and indices
4.1. Indicators, environmental indicators and
environmental performance indicators
The definitions of indicators are particularly confusing [36].
Some specific definitions of indicator in the literature are:
measure of system behaviour in terms of meaningful and perceptible attributes [37]; measure that summarizes information relevant to particular phenomenon, or to reasonable
proxy for such to measure [38]; parameter, or value derived
from parameters, which points to/provides information about/
describes the phenomenon/environment/area with significance
extending beyond that directly associated with parameter
(property that is measured or observed) value [39]; and variable that describes the system, where to variable is an operational representation of an attribute (quality, characteristic,
property) of the system and it represents our image of an attribute defined in terms of the specific measurement or observation procedures [40].
In general, indicators should be able to [41]: (i) assess conditions and trends; (ii) compare across places and situations;
(iii) assess conditions and trends in relation to goals and targets; (iv) provide early warning information; and (v) anticipate
future conditions and trends. So, it is possible to say that desirable indicators are variables that summarize or otherwise
simplify relevant information, make phenomena of interest
visible or perceptible to the managing staff, and are able to
quantify, measure, and communicate relevant information.
Some of those properties are not universal requisites (e.g.
qualitative indicators may be used in some situations), but
a matter of convenience [41]. Most definitions of environmental indicators rule out the possibility of qualitative indicators,
by restricting the concept to numerical variables, either explicitly or implicitly [32,36,37,39,42]. Indeed, it is maintained that
one of the essential functions of indicators is to quantify an
item. Qualitative indicators may be preferred to quantitative
indicators in at least three cases: (i) when quantitative information is not available; (ii) when the attribute of interest is inherently non-quantifiable; and (iii) when cost is a crucial issue,
overwhelming all other considerations. In some cases, qualitative assessments can be translated into quantitative notation.
519
520
1. Environmental Performance Indicators (EPI): specific expression that provides information about an organizations
environmental performance, which are divided into:
Management Performance Indicators (MPI): that provide information about the management efforts to influence an organizations environmental performance;
and
Operational Performance Indicators (OPI): environmental performance indicator that provides information about the environmental performance of an
organizations operations;
2. Environmental Condition Indicators (ECI): specific expression that provides information about the local, regional, national or global conditions of the environment.
The ECls provide information about the condition of the environment. This information can help an organization to better
understand the actual impact or potential impact of its environmental aspects, and thus assist in the planning and implementation of EMS.
It is generally difficult to choose the suitable performance
indicators, as well as to define their suitable number which
can describe thoroughly what one wants to know. Also,
when different specific needs for information are to be fulfilled, then different indicators should be chosen, or built.
In EMS, the classification of indicators recommended by
standard ISO 14031 and the examples of indicators provided
should be considered first, even though they are not complete
or comprehensive [7,21,48,49]. Particularly, an organization
should make a list of indicators by following the general
guidelines of the ISO 14031 standard and linking them to
the corresponding environmental aspects. Table 2 shows an example of a table that can be filled to make it easier the connection between an environmental aspect and the corresponding
indicators.
4.2. Indices
The distinction between indices and indicators is not clear
yet. Regarding this issue, there are two different opinions.
Many authors [32,50,51] put indices on a higher level of aggregation than indicators (Information Pyramid ). Other authors [41] report that indices and indicators differ because of
different complexities of the function by which they are obtained, not because of their hierarchical level. In most cases,
Table 2
Example of a table for linking activities (products or services) and their environmental aspects to the corresponding indicators
Activity/product/service
Combustion
Environmental aspect
Condition
Indicators
OPI
MPI
ECI
NOX concentration
in air
.
.
N, A, E
.
.
.
.
.
.
Another important issue is related to the last point: the assurance of primary-data quality, paying particular attention to the
choice and to the metrological quality of the raw data [44,52,53].
5. Uncertainty of measurement
When you can measure what you are speaking about, and
express it in numbers, you know something about it; but
when you cannot express it in numbers, your knowledge is
of a meagre and unsatisfactory kind. It may be the beginning of knowledge, but you have scarcely, in your thoughts,
advanced to the stage of science
(Lord Kelvin)
The knowledge of the uncertainty is very important because
it implies increased confidence in the validity of the result of
a measurement.
Uncertainty, as defined in Guide to the Expression of Uncertainty in Measurement (GUM e [55]) and in VIM (International
Vocabulary of Basic and General Terms in Metrology e [56]), is
a parameter, associated with the result of a measurement, that
characterizes the dispersion of the values that could reasonably
be attributed to the measurand The parameter may be, for
example, a standard deviation,1 or the width of a confidence
interval; while the measurand is a particular quantity subjected
to measurement that is a set of operations having the object of
determining a value of a quantity [56].
The science of measurement, embracing both experimental
and theoretical determinations at any level of uncertainty in
any field of science and technology, is the metrology (International Bureau of Weights and Measures e IBPM e [57]).
1
The standard deviation of the mean X of n values taken from a population
p
is given by: SX S= n.
521
522
The expanded uncertainty defined above is assumed to provide a high level of coverage for the unknown true value of the
measurement of interest, so that for any measurement result, Y:
Y U True value Y U
For further information see Refs [55,56,58e60].
The evaluation of measurement uncertainty is essential for
the metrological quality of the results. Without such evaluation
the results of the measurements cannot be compared neither
between themselves nor with literature or standard values.
Another important application of the assessment of the
uncertainty is for benchmarking. Benchmarking should be at
the base of EMSs. In fact, it is not only an important tool for
boosting improvements, as it allows to compare and rank
organizations performances, but it also goes beyond the
establishment of benchmarks, standards and norms, since it
investigates the practices that support the benchmark itself
(e.g. [61e64]).
Sometimes, the word uncertainty is interchanged with
accuracy, but they are actually different. In fact, the Accuracy of measurement is the closeness of the agreement between
the result of a measurement and a true value of the measurand and it refers only to systematic error [56]. Therefore,
the word accuracy should not be used for quantitatively describing the characteristics of measuring instruments or other
entities. Even ignoring this point, the term accuracy is a partial contribute of the metrological term uncertainty, which
refers to both systematic and random errors.
6. Case study: the role of uncertainty of measurement in
environmental performance evaluation of municipal
wastewater discharges
Following the trend in the private sector, EMSs have been
introduced in several Local Authorities around the world
Table 3
Indicators that have been used to describe (i) the environmental quality of the environmental compartment hydrosphere and (ii) how it is managed by an
organization [76]
Indicator
Hydrosphere
Water Management
Watershed area (km2)
Main water bodies (No., km2)
Mean flow rate in main rivers (m3 s1)
Artificial water basins and capacity of reservoirs (No., km2, m3)
Water supply sources and abstractions (L s1)
River banks preservation areas (No., % of watershed area, km2)
Indicator of congruity of preservation areas with the Provincial Master Plan (%)
No. of works carried out on artificial reservoirs on a yearly basis
No. of abstractions and flow-rate diversions (L s1)
Environmental quality
Extended Biotic Index (EBI)
Ecological State of the Water Body (according to Italian law n. 152/06, Part 3, Addendum 1)
Ammonium nitrogen concentration (mg L1)
Dissolved Oxygen (mg L1)
E. coli (UFC in 100 mL)
Conformity to regulations for bathing and swimming (%)
No. of complaints per year
ECI
MPI
523
land along the main water bodies compared with the requirements reported on the Master Plan.
Many indicators, instead, have been obtained by actual
measurements, such as the concentration of specific elements
and pollutants in the water body (e.g.: dissolved oxygen and
ammonium nitrogen concentrations). Actually, most indicators
that are commonly used for the characterization of this environmental aspect have been obtained through direct measurements. In Table 4 some examples of performance indicators
referred to the activity management of a sewer system and
the related wastewater treatment plant are reported. They
are classified according to ISO 14031 as follows:
- operational performance indicators (OPI) give information
about the efficiency of the sewer system and the wastewater treatment plant with particular care to the related environmental impacts;
- Management Performance Indicators (MPI) give information about management practices of the sewer system and
the wastewater treatment plant.
By applying and evaluating the proposed indicators to the
specific case of a Municipality, it appeared very clearly that
the compartment water presented some critical elements,
mainly related to the environmental aspect wastewater
discharges.
Particularly, the indicators have shown the existence of
a microbiological contamination of the water body, which
was most probably linked to the discharge of the wastewater
treatment plant. In fact:
- indicator Concentration of Escherichia coli in river
(ECI e CFU/100 mL) (Fig. 1) shows that microbiological
contamination of river is growing between 1999 and 2000:
the limit stated by law (700) is largely overcome (1300).
Escherichia coli (usually abbreviated to E. coli), in fact,
is one of the main species of bacteria that live in the lower
intestines of warm-blooded animals, including mammals,
Table 4
Examples of indicators that can be considered for the environmental aspect defined as wastewater discharges for the activity/service defined as management of
sewers and wastewater treatment plant [76]
Environmental aspect: wastewater discharges
Activity
Indicator
Per capita wastewater discharge into sewers (L per capita and day)
COD concentration at inlet/outlet from WWTP (mg L1)
BOD5 concentration at inlet/outlet from WWTP (mg L1)
SST concentration at inlet/outlet from WWTP (mg L1)
Nitric/nitrous nitrogen concentration at inlet/outlet from WWTP (mg L1)
Ammonium nitrogen concentration at inlet/outlet from WWTP (mg L1)
Phosphorus concentration at inlet/outlet from WWTP (mg L1)
Mass of biosolids that are disposed off yearly (t per year)
Ratio of population equivalent served to total population (%)
Compliance to limits at sampling points (%)
Ratio of No. of actions to No. of sampling (yearly base, %)
Beaches where bathing was declared forbidden by ordinance (No. per year)
Maintenance actions (No. per year)
Complaints concerning malfunctions or nuisance originated by the sewers/WWTP (No. per year)
OPI
MPI
524
Escherichia Coli
1600
1400
ufc/100 ml
1200
1999
2000
2001
1000
800
Law limit
600
400
200
0
Municipalities
Fig. 1. Indicator E. coli for the Municipality considered (shown by the solid arrow).
525
Table 5
Population equivalent (P.E.) in five Municipalities belonging on the same
watershed
Table 6
Values of the State of Treatment Index(SWTI) referring to data reported in
Table 5
Municipality
Sewered
P.E.
P.E. served
by WWTP
P.E. served by
inappropriate
WWTP
Municipality
P.E. served by
WWTP/Sewered
P.E. (%)
SWTI (%)
A
B
C
D
E (implementing an EMS)
9073
452
1355
997
3764
8478
452
186
997
3205
227
0
0
686
1100
P.E. served by
inappropriate
WWTP/P.E.
served by
WWTP (%)
A
B
C
D
E (implementing an EMS)
93.4
100.0
13.7
100.0
85.1
2.7
0.0
0.0
68.8
34.3
90.8
100.0
13.7
31.2
50.8
P:E: served by inappropriate WWTP
100
Tot P:E: served by WWTP
Municipality E
Municipality D
Municipality C
Municipality B
Municipality A
0
10
20
30
40
50
60
70
80
90
100
110
Fig. 2. SWTI for Municipality E (implementing an EMS) and for other Municipalities in the same watershed.
P:E:
3000
2500
Type
1
A
2
A
3
NA
4
NA
5
NA
6
NA
Total P.E. served by WWTPs
A appropriate; NA not appropriate.
BOD5 Load
(BL, kg d1)
P.E.
66.00
60.30
32.40
18.00
12.00
3.60
1100
1005
540
300
200
60
3764
2000
1500
1000
500
0
32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68
SWTI (%)
Variable: Municipality E, wc(BL) = 10%, PDF: Normal
Chi-square Test = 44,51302, gl = 8 (adgiust.), p = 0,0586
3000
2500
2000
1500
1000
500
0
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85
SWTI (%)
Variable: Municipality E, wc(BL) = 20%, PDF: Normal
Chi-squareTest = 15,45378, gl = 8 (adgiust.), p = 0,0592
3000
2500
No. of obs
WWTP
BL
0:06
No. of obs
No. of obs
526
2000
1500
1000
500
0
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85
SWTI (%)
Fig. 3. SWTI frequency distribution in Municipality E according to different relative standard combined uncertainties wc(BL) 5%, 10% and 20%.
The BOD5 load (BL, kg per day), which is the startingpoint raw datum used to calculate P.E., is obtained from:
BL BQ
527
7. Conclusions
In the case study described earlier, the number of served
population equivalent of the different Municipalities which
have been evaluated, can be highly affected by uncertainty
when based on BOD5 measurements, while it is often and
Municipality E
wc(BL) = 5%
Municipality D
wc(BL) = 10%
wc(BL) = 20%
Municipality C
Municipality B
Municipality A
0
10
20
30
40
50
60
70
80
90
100
110
STI U
Fig. 4. Representation of SWTI U, where U is the expanded uncertainty that affects SWTI, according to three scenarios on the uncertainty wc(BL) which may
affect BOD5 measurements (5%, 10% or 20%).
528
References
[1] International Standard Organisation. ISO 14001 e environmental management systems e requirements with guidance for use. International
Standard Organisation; 2004.
[2] International Standard Organisation. ISO 14031e environmental management e environmental performance evaluation e guidelines. International Standard Organisation; 1999.
[3] Johnston A, Hutchison J, Smith A. Significant environmental impact evaluation: a proposed methodology. Eco-Management and Auditing 2000;
7:186e95.
[4] Pojasek RB. How do you measure environmental performance? Environmental Quality Management 2001;79e88. Wiley Periodicals.
[5] Dias-Sardinha I, Reijnders L. Environmental performance evaluation and
sustainability performance evaluation of organizations: an evolutionary
framework. Eco-Management and Auditing 2001;8:71e9.
[6] Ammenberg J, Wik G, Hjelm O. Auditing external environmental auditors e investigating how ISO 14001 is interpreted and applied in reality.
Eco-Management and Auditing 2001;8(4):183e92.
[7] Butelli P, Marchesi R, Perotto E. Certificazione ambientale e riferibilita`
delle misure. In: Maggioli, editor. Atti dei seminari di Ricicla 2001,
Rimini Fiera, 26e29 settembre 2001; 2001. p. 126e33.
[52]
[53]
[54]
[55]
[56]
[57]
[58]
[59]
[60]
[61]
[62]
[63]
[64]
[65]
[66]
[67]
[68]
[69]
[70]
[71]
[72]
[73]
[74]
[75]
529
530
[76] Manly BFJ. Randomization and Monte Carlo methods in biology. In:
Statistical sciences series. Chapman & Hall Texts; 1998.
[77] Saltelli A, Chan KP, Scott M. Sensitivity analysis. In: Probability and
statistic series. John Wiley & Sons Publishers; 2000.
[78] Cox M, Harris P. The GUM and its planned supplemental guides.
Accreditation and Quality Assurance 2003;8:375e9.
[79] Butelli P, Menegozzi S. La valutazione dei carichi inquinanti addotti dai collettori fognari: caratterizzazione delle acque reflue ed incertezza di misura. In:
Atti del IV Congresso Metrologia & Qualita`, Torino; 2005. vol. 1. p. 48e52.
[80] Bottazzini N, Butelli P. Qualita` del dato analitico, Atti del 55 Corso di
Aggiornamento in Ingegneria Sanitaria, Il nuovo laboratorio per lambiente. Politecnico di Milano: DIIAR; 2001.