Professional Documents
Culture Documents
Overview25
Dissemination for Methods Using Calibration Standards Produced with
Concentration Values Linked Directly to SI units: Ambient Air.26
2.6.3 Dissemination for Methods that Use CRMs for Quality Control and Quality
Assurance: Ambient air...28
2.6.4 Dissemination Using Primary Methods: Ambient Air30
Overview..31
Summary of Comparisons Organised by the EC Joint Research Centre..31
International Comparisons carried out by the CCQM and Associated
Organisations.32
2.7.3.1 Background..32
2.7.3.2 International Comparisons of Ambient-Air Calibration Standards.33
5.6
5.7
5.9
Chapter 6:
6.1
6.2
6.3
6.3.1 BTEX65
6.3.2 VOC Ozone Precursors...66
6.4 Examples for PM Constituents: Heavy Metals and EC/OC.68
6.4.1 Heavy Metals.68
6.4.2 EC/OC.69
6.5 Example of PM10 and PM2.5 Comparisons.70
References.71
Annex 1: Organisation of Intercomparison Exercises for Gaseous Air
Pollution for EU National Air Reference Laboratories and
Laboratories of the WHO EURO Region.72
ANNEX 2: Background to the Requirements for Quality Systems from the
Previous Directive 1996/62/EC81
1.2
Air quality across the whole European Union (EU) is a field where the European
Commission (EC) has been proactive in defining a comprehensive strategy and an
implementation methodology. Recently, this has resulted in the plans and initiatives
developed under the Clean Air for Europe (CAFE) programme. This Programme have
also involved the publication, or the revision, of a range of EU directives, concerned
both with ambient air quality and atmospheric pollution emissions to the atmosphere.
The directives currently of most relevance to this document are:
2008/50/EC The new Directive of the European Parliament and of the Council on
ambient air quality and cleaner air for Europe (Official Journal of the European Union
L152 11.6.2008).
2004/107/EC - 4th Daughter directive: relating to arsenic, nickel, cadmium, mercury
and polycyclic aromatic hydrocarbons in ambient air;
In addition, for completeness, it should be considered that this document covers the
requirements of the previously published directives listed below, which will be repealed
by June 2011 as the new Directive is required to be transposed in all Member States:
1996/62/EC - Air Quality Framework directive on ambient air quality assessment and
management;
1999/30/EC - 1st Daughter directive relating to limit values for sulphur dioxide,
nitrogen dioxide and oxides of nitrogen, particulate matter and lead in ambient air;
2000/69/EC - 2nd Daughter directive relating to limit values for benzene and carbon
monoxide in ambient air;
2002/3/EC - 3rd Daughter directive relating to ozone in ambient air;
The above directives all on ambient air quality, and particularly the recent Directive
2008/50/EC, have a number of general objectives, which may be summarized as:
- define objectives for ambient air quality to avoid, prevent or reduce harmful effects on
human health and the environment as a whole,
- assess ambient air quality in Member States on the basis of common methods and
criteria,
- obtain information in order to combat air pollution and monitor long-term trends,
- ensure public information,
- maintain good air quality and improve in other cases, and
- promote cooperation between Member States in reducing air pollution.
The requirement for ensuring measurements of known measurement uncertainty that
are comparable across EU is crucial for ensuring that appropriate abatement
measures are triggered, where necessary, and that a level-playing field exists for any
potential enforcement of the provisions of the ambient air quality Directives.
1.3 National Reference Laboratories: What are they? How are they
(NOTE 2: In Directive 2008/50/EC, and the previous ambient air directives listed
above, the term Community-wide intercomparisons is used. It is felt, however, that
the word intercomparison is editorially unnecessary in this document in English
vocabulary, and instead the word comparison is used, whenever this term in the
directive is not being specifically quoted.)
A number of points may be concluded and emphasized concerning Article 3 of
Directive 2008/50/EC. Many of these might already be clear to the reader of this
document. However, they are emphsized below for completeness and in order to
provide a better uniformity of the intended concepts and conclusions across the
AQUILA Network and other relevant organisations:
(i) It is the responsibility of the competent authority of the specific Member State, to
designate or appoint relevant organisation(s) to carry out all the tasks listed in (a)
(f) above, where these competent authorities in each country are as defined
above generally national government ministries or national agencies.
(ii) It may not be practical for all of the tasks listed above to be carried out by one
organisation in many Member States. For example, in some of the larger Member
States there may be different organisations that cover different pollutants, in others
all may be covered by one organisation. In addition, some of these tasks are
broader or different than those required of an NRL (see below). This may also
allow implicitly for subcontracting of certain tasks to other organisations or
individuals, but it clearly leaves the designated body with its responsibilities
outlined above and discussed in more detail later in this Document. These issues
are all for the competent authority to define.
(iii) Some of the above tasks comprise mainly scientific and technical experimental
work associated with the requirements of Annex 1 Section C, whilst others do not.
This enables us to differentiate whether there are requirements for the designated
bodies for a particular task to be accredited to the EN ISO 17025 standard
discussed in Annex 1 or not application of EN ISO 17025 is relevant to those
organisations that have involvement with scientific and/or technical experimental
quality assurance work, and not to the other tasks.
(iv) The competent bodies that are designated for the tasks involving the scientific and
technical activities associated with quality assurance and quality control of the
accuracy of the results of all the regulated ambient-air pollutants at a
national/Member State level and that are involved in participating in, or
coordinating, Community-wide quality assurance programmes being
organised by the Commission (Article 3) should be considered as NRLs
(v) The NRL(s) in a given Member State are also those that take part in the
Community-wide intercomparisons covering one or more of the pollutants
regulated by the directive, as stated in Annex 1 Section C, and these require
accreditation to the EN ISO 17025 standard. The exact scope of the accreditation
required is discussed below.
(vi) There are a number of ambient air pollutants that are covered by Directive
2008/50/EC (and others covered by directive 2004/107/EC the 4th daughter
directive). It is therefore possible that all the scientific and technical quality
assurance tasks that are required to be implemented might be carried out by
different organisations within a Member State, and then there will be more than
one designated NRL in any Member State to cover all of these. This, as noted
above, is the responsibility of the competent authority of a Member State to define.
(vii) There may be a multi-level or tiered system of quality assurance and quality
control to ensure the accuracy of the measurements in a given Member State. This
is acceptable within the requirements of the directive(s), and this is again a
decision of the competent authority in that Member State.
(viii) Where there is a multi-level or tiered quality assurance system for ambient air
quality measurements relevant to the Directive 2008/50/EC (and by implication to
directive 2004/107/EC), the definition and designation of an NRL, and the issues
related to accreditation to the EN ISO 17025 standard apply only to the national
(highest-level) organisation that also participates at the Community-wide
intercomparison, unless national transposition of the Directive requires otherwise.
However, in this case it should be understood that the competent authority and/or
body still has certain responsibilities for ensuring the accuracy of all relevant
measurements within the Member State as discussed further below (Section
3.4).
(ix) The competent authority of a given Member State may, in principle, designate an
organisation outside of that Member State, where it can be assured of the quality
and accuracy of the measurements at a national or other level. This might occur if
there is no competent body with suitable expertise within the Member State.
(x) The competent authorities in Member States are, as stated previously, generally
national government ministries or national agencies, and these may choose not to
designate permanently any given organisation or laboratory as the competent body
for a given range of the ambient air activities defined in Article C. Instead it may
wish to retain this designation formally for itself. This may be, for example, because
it allocates its scientific and technical, and other tasks, to different organisations
regularly on a competitive-tender basis, and thus no one organisation remains
permanently responsible. In this case it is anticipated that this competent authority
will designate the organisation currently responsible for a range of quality
assurance and quality control activities, and where possible these therefore
become NRLs, and thus participate in AQUILA and the other required activities,
and these will need to gain accreditation to the EN ISO 17025 quality-assurance
standard with a scope that is described in detail in Section 3.3 of this document.
3. To provide scientific and technical advice, where required, to NRLs that are less
well developed and to other organisations (European Environment Agency, World
Health Organization etc);
4. Provide appropriate scientific and technical advice to the European Commission to
support current EU legislation and aid in the development of future policy; This
includes, for example, the provision of quantitative performance data on new
monitoring methods under a range of realistic operating conditions, in order to
further environmental protection by substantially accelerating the acceptance and
use of improved and cost-effective technologies that provide equivalent results
5. Provide technical advice to the Competent Authorities in the EU Member States, in
order to assist with the prompt dissemination of EU policies and monitoring
requirements, and ensure up-to-date, valid, and harmonised implementation of
these, and where needed, provide interpretation;
6. To raise the profile of the roles and responsibilities of AQUILA and the NRLs, in
order to make greater inputs and thereby improve, where required, scientific and
technical activities within Member States;
7. Coordinate, or contribute to the coordination of, international and national intercomparison exercises carried out for the purposes of demonstrating the
harmonisation of ambient air-quality measurements across Europe;
8. Participate, where possible, in European standardization activities in the field of
relevant ambient air-quality measurements, and/or provide technical advice to
these. This may be done by means of common representation of AQUILA
members on the relevant CEN Working Groups, and/or through recommendations
of the AQUILA group for consideration by the relevant CEN Working Group.
9. Act as a forum for the collation of practical experiences on published standards so
that future revisions are based on sound and up-to-date data.
The objectives listed above represent the main technical activities that the AQUILA
Network has been, and is involved with. However, the AQUILA activities become
broader in detail as time progresses. For example, the AQUILA Group is now
considering the best methodology for handling and presenting monitoring data at close
to the detection limits to ensure a harmonised means of applying this across Europe.
The subsequent Chapters in this document provide background information on the
selection and the responsibilities of National Reference Laboratories that make up the
AQUILA Network, and also their requirements for accreditation as used in the context
of the European ambient-air directives summarized above. The subsequent Chapters
also provide details on the main scientific and technical requirements that the AQUILA
Members must conform to, and some of the most relevant activities of AQUILA
Network itself up to the present time are also discussed.
10
11
12
SI Units
c
o
m
p
a
r
i
s
o
n
s
(Inter)national (Primary)
Standards
Secondary Standards
Work Standards/In-house
Standards
Measurement/Calibration
uncertainty
Figure 2.1: Schematic of how the Hierarchy of Traceability is achieved, and the
Resulting Measurement Uncertainties
SI system of units
prim ary direct m ethods
(e.g. coulom etry, FPD)
pure m aterials
prim ary ratio m ethods
(e.g. IDM S)
13
14
NMI in a given country may seek to identify and to designate another laboratory as its
representative for a certain range of measurement activities, where it does not have
the expertise, or the capacity to maintain itself in that country.
The national standards developed in any specific NMI are generally maintained and
retained within that institute, and not disseminated or distributed to outside customers
or other interested organisations. Instead, there are generally two different
mechanisms for disseminating the traceability from a nationally established primary
measurement standard or reference measurement method. This is by:
(1) Bringing the equipment, a measurement method, or some form of calibration
artefact to the NMI, where it is calibrated with respect to the national standard by
means of experimental comparisons that have known performance and
measurement uncertainty.
(2) The NMI producing and marketing calibrated artefacts or other measurement
techniques, that has been calibrated with respect to the national standard held by
the NMI, where the calibration has been shown to be stable over time. This then
can therefore be provided for application by the end user externally to the NMI.
These two mechanisms may give rise to nationally traceable calibrations that have
different measurement uncertainties. Both of these mechanisms are used by NMIs
worldwide. The mechanism selected depends on the quantity being disseminated and
also on the optimum way that achieves the required traceability with the necessary
measurement uncertainty. A schematic diagram of the general hierarchy of traceability
applicable to many measurements is shown in Figure 2.1 above, indicating clearly that
the measurement uncertainties increase as the measurement becomes further from
the source of traceability.
In addition, it is important to recognise that many NMIs (or other laboratories at a
national level designated for a particular field of measurement expertise) participate
regularly in comparisons, usually between each other. This is necessary in order to
demonstrate the international comparability and accuracy of the national standards
they maintain. Some of the methods for doing this are discussed in Section 2.7 below.
Section 5 below summarizes the work that NMIs are carrying out in certain countries in
the field of ambient air-quality measurements, and in particular for the provision of
traceability to nationally maintained or internationally recognised standards, and/or
using internationally-accepted Certified Reference Materials. This procedure for
disseminating traceability is similar in principle to that shown in figure 2.1, and this is
explained in more detail in Section 2.5 below.
2.4 Certified Reference Materials and their Roles in Quality Assurance and
15
results between one organisation or one country and another, with certain limitations,
particularly where no fully traceable method exists. For example, in general, CRMs can
only be considered calibrants where they are used in a manner such that they exhibit
the certified value for the quantity of concern in the method being used, with no other
effects due to their different composition sample matrix effects etc. Thus, where CRMs
are used properly for calibrations they provide traceability but where they are used in
quality control and quality assurance only they do not strictly provide traceability.
A Reference Material (RM) may be defined as:
Material, sufficiently homogeneous and stable with reference to specified properties,
which has been established to be fit for its intended use in measurement, or in the
examination of nominal properties (of a sample);
This emphasizes that certain of the properties only may be homogeneous and stable.
It specifies nothing about the other properties of the RM, or its matrix. This therefore
means that it may be only applied where the effects of not knowing these other
properties, or the effects of the matrix material(s), are not significant for the validity of
the analysis being used. This also constrains how they should be applied.
The activities whereby Reference Materials are produced characterised and certified is
one of the key activities in improving and subsequently maintaining a comparable,
coherent, stable, and accurate system worldwide for nearly all types of measurements.
These result in what is known generally as a Certified Reference Material (CRM). The
definition of this may be considered as:
Reference material, accompanied by documentation (usually in the form of a
recognised certificate) issued by an authoritative body using valid certification
procedures and providing one or more specified values for specified properties
with associated uncertainties and with specified traceabilities.
The definition indicates some of the differences between nationally or internationally
traceable calibration standards that are discussed in Section 2.3 above, and RMs and
their associated CRMs:
-
The properties and limitations summarized above, are also applicable to the CRMs
used in the validation of the measurements of ambient air quality that are regulated by
EU directives, as discussed in Section 2.5.3. They are rarely used as calibrants in
these ambient-air pollutant cases.
16
2.5.1 Overview
This Section outlines some of the methods by which nationally traceable calibrations
and/or calibration standards and Certified Reference Materials are realized in the field
of ambient air quality, in the context of the requirements of the EU directives:
-
The different methods of dissemination of these standards are also summarized below
in Section 2.6, which include the distribution of what are sometimes described as
Certified Reference Materials that are traceable to these national standards.
The use of other Certified Reference Materials, that are internationally recognised
and used widely across different countries, is discussed separately in Section 5.3
below.
The methods by which nationally traceable calibrations and/or calibration standards
are developed and maintained at a national level within the scope of this document fall
into a number of different categories. These cover calibration standards that are:
(1) Prepared in a manner that is traceable directly to SI units though mass, volume
(length), flow (mass and time) etc. These usually apply to calibration standards for
certain pollutant gases, although there are different means of realizing and
disseminating these (see Section 2.5.2 below);
(2) Not able to be produced in a manner that is directly traceable to SI units but
traceability can be achieved by means of the realization of a potentially absolute
method such as optical photometry, when implemented under controlled conditions
(see Sections 2.5.3 & 2.5.4 below);
(3) Not able to be realized in a traceable manner, and not able to be realized as a
primary or absolute method, but it is realized by convention or by definition as the
reference method, preferably having the smallest measurement uncertainties
achievable using that method (see section 2.5.5 below);
2.5.2 Methods for the Production of Calibration Standards Traceable
to SI units: Ambient Air
2.5.2.1 Overview
Directly
There are a number of methods for the preparation of calibration standards that are
considered as traceable directly to the SI system of units. These, which are described
in published EN ISO standards, are summarized below. They have been used by NMIs
to develop and maintain their national/primary standards, and several international
comparisons have been carried out between NMIs worldwide. The most important of
these comparisons are summarized in Section 2.7.3, which demonstrate
comprehensively the level of the comparability and accuracy of these nationallyrealized primary standards that have been developed and maintained using these
primary methods.
There are, however, a number of quality assurance requirements that are absolutely
essential to implement if accurate and comparable standards are to produced and
maintained. Many of these are specified in detail in the relevant EN ISO standards
discussed below, but one common and important issue in the quality control is the
17
requirement in all the preparation methods for diluent gases with sufficient and
monitored purity and this is discussed in Section 2.5.2.3 below
2.5.2.2 Preparation of Gaseous Calibration Standards
The most convenient calibration standards that can be produced to provide traceability
directly to SI units at a national level are those for the gaseous pollutants covered by
the 2008 EU Directive:
(a)
(b)
There are a number of primary methods that can be used for preparing these
standards that are considered directly traceable to SI units as listed in Section 2.2 (a)
above (where traceability is achieved directly to one of the base units of the SI system
such as mass or length (volume), as well as to one or more of the derived SI units,
such as pressure). These are as referred to in Section 2.5.1 bullet (1).
These have been developed into international standards (norms) through the
International Standardization Organisation (ISO), and mainly through its Technical
Committee TC 158. This has produced a suite of ISO standards specifying different
methods for preparing and validating the prepared gas standards, as summarized in
Table 2.1, presented for the example of NOx calibration standards.
Outlines of the different preparation techniques listed in Table 2.1 are summarized
below for clarity and completeness. For more detailed information, the relevant ISO
standards given in Table 2.1 should be consulted:
One of the methods used worldwide to prepare gas standards at all levels of
concentration, for many different species, is to use gravimetry to weigh specific
pollutant gases together with relevant matrix/diluent gases into high-pressure
cylinders, and then to dilute these gas mixtures to the required low concentrations
for ambient air concentration calibrations, as required. A number of NMIs and other
designated laboratories, carry out this process. An advantage is that these gas
mixtures can be prepared for use at the low concentrations required. A
disadvantage is that the lower the concentrations prepared the more likely the gas
standard is to be unstable for certain pollutant species. It is important to recognise,
however, that comprehensive quality assurance and quality control procedures are
required if valid standards are to be produced, and this becomes increasingly
difficult at the low concentrations required for ambient calibrations. This method is
an example of where gas standards are prepared that are directly traceable to the
SI mass unit, via a number of accurate weighings (gravimetry), as stated in Section
2.5.1 bullet (1).
A further method, usable only for some pollutant species and concentration ranges,
is to employ permeation tubes. These permeation tubes are designed so that a
given mass of pollutant gas permeates from the tube per unit time, depending on
the temperature at which the tube is maintained. Then, if this gas is mixed into a
stream of diluent gas of known flow rate, a known concentration gas standard may
be produced. This requires that the mass loss of the permeation tube is
determined, and the flow of the diluent gas is measured. The mass loss is thus
traceable to the SI unit of mass, and the dilution process is traceable either to
pressure measurement, or it may alternatively be calibrated using gravimetry.
18
Method
NOa
NO2
Description
Cylinder
ISO 6142
ISO 6143
Permeation tubes
ISO 6145-10
Static dilution
ISO 6144
Dynamic dilution
ISO 6145-6/7
19
The different methods used in different countries for disseminating traceability from
these national standards to the individual network sites are also summarized - see
Section 2.6 below.
NOTE 4: It is worth clarifying in the context of definitions, that gas standards that
are prepared by the primary methods discussed above, provide calibration standards
that are directly traceable to one or more of the international units of measurements,
(and hence may be used to provide robust calibration methods when applied
correctly), could fall within the definition of Certified Reference Materials (CRMs) as
defined above.
The gas standards produced by these methods may therefore sometimes be referred
to as CRMs. However, for the purpose of this document, and in relation to ambient air
calibrations, a distinction is made between these calibration standards described
above for ambient air, and the specific and different actual CRMs discussed below.
These CRMs are employed in a different manner within ambient air monitoring
applications.
2.5.2.3
Purity Requirements for Diluent Gases, Calibrant Gas Species, and Zero
Gases
In all the above gas standard preparation techniques, there are critical requirements to
carry out analyses of the purity of all components used for the standard. There are a
number of reasons that measurements of impurities in the diluent and calibrant
components of the standard are essential, in order to ensure the accuracy of the
calibration and/or ensure the stability of the mixture with time:
(a) Impurities in the diluent gas may react with the calibrant gases and an incorrect
concentration of the prepared calibration gas will result. Examples include:
- trace levels of oxygen in the nitrogen diluent used to prepare nitrogen monoxide
standards, which will result in conversion of some of the nitrogen monoxide to
nitrogen dioxide;
- water vapour levels in diluent air or nitrogen when used to prepare sulphur
dioxide standards may result in mixtures that are unstable with time;
- when there is a significant concentration of the calibrant in the diluent gas, then
this will increase the concentration of the resulting calibration gas unless
measured and corrected for. This is most important for low concentrations.
(b) Impurities in the pure calibrant gases will give rise to incorrect calibrations unless
the impurities are analysed and corrected for. One example is the presence in
nitrogen monoxide of impurities comprising other oxides of nitrogen and nitrogen
itself.
In addition, it is clear that presence of the trace calibration gas in all zero gases used
to provide the zero response of an analyser, or any other actively interfering gaseous
species, will clearly result in incorrect zeroing of the analyser.
It is therefore clear that all components used in the preparation of all calibration gases,
and wherever zero gases are employed, must be analyzed with sufficient sensitivity
and accuracy or incorrect results may occur. These activities should be one part of the
NRLs accreditation to the EN ISO 17025 standard, wherever relevant. However, it is
not necessary for the NRL to perform such analyses themselves, but these should
then be obtained from a suitable source. Certified zero reference gases (usually of
nitrogen or synthetic air) are available commercially with suitable levels of purity.
2.5.3 Methods using Certified Reference Materials, and their Role in Quality
Assurance and Quality Control: Ambient Air
20
The methods discussed in Section 2.5.2 above provide robust means for providing
calibration standards, with applications to ambient air quality, directly traceable to one
or more of the international system of SI units. However, in some cases it is not
possible to produce calibration standards this way. Then internationally recognised
CRMs, where certain of their properties are well validated, are employed instead. Such
CRMs are generally prepared and certified by one or more expert laboratories
worldwide, where these have a range of different independent techniques to certify
such CRMs.
Those relevant to the monitoring of ambient air pollutants that are covered by the EU
Directive 2008/50/EC and the 4th daughter directive 2004/107/EC are discussed below:
(i) There are requirements in the above directives for the analyses of the heavy
metals lead, cadmium, arsenic and nickel. These are required to be monitored
using PM10 ambient-air manual samplers onto specified filter media in the field,
followed by laboratory analyses using Inductively-Coupled Plasma Mass
Spectrometry (ICP-MS) or by Graphite Furnace Atomic Absorption Spectroscopy
(GF-AAS), using comprehensive procedures that are specified in European
standard EN 14902. These procedures are examples of measurement methods
that are not directly traceable to the international system of SI units, but to a
prescribed procedure with partial traceability, and operated under tightly controlled
conditions, as prescribed in the EN standard. In this, traceability of the analysis is
achieved by preparing gravimetrically solutions of the metals concerned with
known concentrations. These are traceable to SI units via the national or the other
laboratory that prepared or certified them. However, they are used in these
applications for the calibration of the analytical part of the method only. The results
of the extraction/digestion of the ambient air sample cannot be made traceable in
such a manner. Instead, some CRMs are available to check that the
digestion/extraction remains under control. The CRM is thus used to check or audit
the validity of the procedure during use - to demonstrate that certain aspects of the
procedure are valid whenever it is implemented. Examples of relevant CRMs are
given in EN 14902:
NIST SRM 1648a (subsequently 1648b has been issued) prepared and
certified by the National Institute of Standards and Technology (NIST) USA;
SRM NIES No.28 certified by National Institute for Environmental Studies
(NIES), Japan;
More recently, work is being completed at the EU Joint Research Centre, Institute
of Reference Materials and Measurements (IRMM), Geel Belgium, on the
preparation of two new CRMs (one for Heavy metals, and the other for PAHs).
These will have improved characteristics when compared to the CRMs of NIST
SRM1648 & NIES SRM No.28 listed above, and to the equivalent NIST SRM for
PAHs (NIST SRM 1649a), and should be used widely for quality control and
audits of EU directive-regulated air pollutant measurements when they
become available.
There are certain limitations in the applications of any SRMs for these
analyses of heavy metals (and other analytes), and some are summarized below.
These limitations are given here to illustrate some of the differences between
CRMs and the calibration standards discussed in Section 2.5.1 above, and also to
show why they are generally used as quality checks and audits of the analytical
procedures, rather than being applied directly for the calibrations themselves.
These limitations include:
The metals are contained in a solid matrix (urban dust), but this is never identical
to the particulate matrix that constitutes the real sampled PM10 particulate.
21
Neither is this matrix material contained on a filter medium as is the sampled PM,
and it may therefore not be digested in an identical manner to that of the real
sampled PM;
The metals are usually present in different chemical forms to those routinely
found in the ambient air, which may not be digested in an identical manner;
The masses of the metals contained within a representative homogeneous
sample of the SRM that must be used for the analyses are generally not very
similar to those masses that are in the real atmospheric samples on filter media;
The uncertainties assigned to the certified concentrations of the metals in the
CRM are generally larger than required ideally if used as calibrants (and they are
also not ideal as calibrants for the reasons above). They would therefore not
satisfy the measurement uncertainty requirements of the relevant ambient air
directives;
In addition, it should be understood that these CRMs are certified using a fixed and
finite amount of original material (usually solid in these cases), and thus this
represents a fixed batch size of the specific CRM. When this is used up, another
CRM must be obtained and another certification carried out afresh where different
values will be assigned (In contrast to the realizations of the gas standards
discussed in Section 2.5.1 above, that can be realized with respect to SI units at
any given time).
(ii) There are requirements in the 4th daughter directive for the monitoring of a range of
olycyclic aromatic hydrocarbons (PAHs), including benzo(a)pyrene (BaP). These
are required to be sampled and analysed according to the standard EN
15549:2008, which is to be considered as the Reference Method. The initial
requirement in the standard is to extract the PAHs collected on ambient particulate
filters using specified solvents and extraction techniques. The efficiency of this
extraction is known as the recovery, of the PAHs. There are then requirements
within the standard to set up calibration curves for specified analysis techniques,
with five different known concentrations of the PAHs for the analyses, using
traceable solutions of one or more PAHs dissolved in specified solutes. There is a
requirement also assess the efficiency of this recovery process using a CRM.
Currently the CRM recommended in the standard is that produced by NIST USA
(NIST 1649a). However, others may be used. The CRM is again used as a quality
check or audit check and is not the primary source of traceability in these PAH
measurements.
The limitations of CRMs for these types of PAH measurements are the same as
those given in Section 5.2 (i) above (except that the concentration of the BaP in the
CRM is certified to an accuracy of ~4%, which would be a satisfactory accuracy for
its application as a calibrant).
(iii)There are also certain requirements in the 4th daughter directive for monitoring the
concentrations of mercury in ambient air. The main requirement, however, in this
directive is for measurements of total gaseous mercury. This is covered by a CEN
standard that is discussed below in section 2.5.3, and no CRMs are available or
required for this. However, the CEN standard specifies that calibration must be
performed by means of mercury vapour contained in a vessel at a known
temperature and using an empirical equation to define the relationship between its
temperature and concentration. Where there are requirements to monitor mercury
and its compounds in particulates, the same CRMs as given in Section 5.2 (i)
above may be used, and the same limitations on their applicability apply, and
22
hence they are being used as quality assurance checks and/or audit samples,
rather than being used for primary calibrations.
A schematic of the means by which the traceability and hence the accuracy of these
methods for the analyses of heavy metals and PAHs on particulate samples is
achieved in general, including the use of primary calibrations with gravimetrically
prepared solutions and CRMs, is shown in Figure 2.3 below.
Analysis
maintained by
Sample Preparation
Analytical Laboratory
Extraction
As for PM10
Sampling
Matrix-matched CRM
Internal Standards
Analytical CRM
23
The mechanism for dissemination of the traceability of the results obtained by these
SRPs is outlined in Section 2.6.3.
The BIPM organisation is involved in carrying out comparisons of the SRPs maintained
by laboratories worldwide, within the auspices of the CCQM. These are used to
demonstrate their comparability as primary methods when realized by the different
institutes that maintain them (see Section 2.7).
2.5.5 Methods not Traceable to SI Units or Primary Methods, Realized by
Convention or by Definition as Reference Methods: Ambient Air
As noted in Section 2.5.1 bullet (3), there are certain calibration standards and
calibration methods that are not required/not able to be developed and realized with
traceability, directly or indirectly to the SI system of units. Examples of most relevance
to measurements of ambient air pollutants currently regulated under the EU directives
are:
(i) The manual methods for particulate monitoring of PM10 and PM2.5 specified in the
European standards EN 12341 and EN 14907 are required to be established and
maintained at a national level by the relevant NRL. These two standards specify
methods that cannot be considered as traceable to the SI system of units, in a
rigorous manner. The sampling is by a method defined by convention, but the
mass measurement can clearly be defined as traceable to SI Units. However, the
main problem is that the measurand itself is not well defined as the particulates
sampled will all have different shapes sizes and chemical compositions.
Notwithstanding this, as is well known, these two European standards have been
specified as the Reference Methods in EU Directive 2008/50/EC. This therefore
imposes a requirement for these standards to be used as references against which
other measurement techniques used for monitoring PM10 and PM2.5 for EU
regulatory purposes, may be compared, and the results must be demonstrated to
be equivalent to be accepted for the purposes of reporting results to the EC. This
concept of the demonstration of equivalence with is specified in a document
prepared by the EC (Reference 4) and involves comparisons in the field of any
other method for monitoring PM10 or PM2.5 with the relevant reference method.
The above standards therefore represent reference methods that are specified by
definition and by convention, with no real traceability to the SI system of units (with
the exception of a requirement to determine the mass flow of air through the
monitor). Figure 2.4 shows the principle whereby a limited level of traceability is
achieved using these reference methods for monitoring particulates. This inability
to achieve traceability is partially as a result of the complex variable and ill-defined
size, shape and composition, of the particulate matter being monitored.
24
Classifier Dimensions
Filter Conditioning
maintained by
Filter Weighing
Calibration Laboratory
Secondary Standards
PS of Length
PS of Flow
PS of Temperature,
Humidity
PS of Mass
25
before, the NRLs identified in each country will generally NOT comprise the same
laboratory for all of these, because of the large breadth of expertise that would be
required.
Moreover, the dissemination mechanisms will depend on the number of sites/sampling
locations that are required and present in a given country (ranging from less than 10 to
more than 1000 locations). In addition, the dissemination mechanisms for these
different air pollutant species will be different in many cases.
The dissemination procedures should therefore be designed differently in each case to
accommodate these issues. Examples of these dissemination mechanisms for the
different types of ambient air pollutants grouped according to the way traceability is
developed, as given in Section 2.5.1 bullets (1) (3), are given below.
It is also useful to emphasize that there is a general requirement, irrespective of how
dissemination is realized, according to Directive 2008/50/EC Annex 1 section C:
These (designated NRL) laboratories shall coordinate at a national level the
appropriate realization of the Reference Methods and the demonstration of
equivalence of non-reference methods;
However, in practice the competent authority in a Member State may choose to define
another organisation to carry these activities out.
2.6.2 Dissemination for Methods Using Calibration Standards Produced that
have Concentration Values Linked Directly to SI units: Ambient Air
This is the most common situation in the context of ambient air pollutants regulated by
the EU directives, since as discussed above in Sections 2.5.1 & 2.5.2, this situation
covers all the gaseous species (except gaseous mercury), where calibration standards
may be produced traceably to SI units, using a number of different methods as shown
in Table FF. However, even in these circumstances there are nevertheless, in practice,
somewhat different ways of disseminating national traceability, depending for example
on the number of network sites within a given country. Currently there are broadly two
methods in use:
(a) The use of a single QA/QC Unit for all of the network sites covering all these
gaseous pollutant species within the country. This then becomes by definition the
single NRL for these monitoring activities, and the traceable calibration standards
available to this NRL can be used directly at each monitoring site, without any
additional steps in the traceability chain. This is generally carried out using
calibration standards produced and certified as discussed above in Section 5.2.
Alternatively the NRL may utilize traceable standards from another organisation
and then certify other mixtures in a manner that is traceable to these. Where this
latter is carried out, this certification process should be accredited to EN ISO 17025
as a calibration accreditation, to ensure the accuracy of the certified calibration
standards used. This procedure maps closely with the mechanism shown in Figure
AA. This structure means for example:
- The number of sites may be large, and this requires relevant expertise to be
available in a large organisation with a skilled workforce, and also that a large
amount of field work is required that encompasses all of the network sites at a
national level;
- There is little requirement to audit the performance of any staff at the individual
network sites since this should be all in the control of one laboratory that must
have traceable standards, and should have appropriate accreditations, to
carry out the QA/QC activities in the field - as specified in the relevant CEN
ambient air standards;
26
E N -IS O 1 7 0 2 5
C a lib r a t io n
C IP M M R A
C a lib r a t io n
L a bCo aralib
tora
r yt io n
L a bCo aralib
tora
ryt io n
L a bCo ar alib
torry
a t io n
L a b o r a to r y
N a tio na l
H a r m o n iza t io n
N a tio na l
R eference
L a b o ra to r y
A ud its
R e g io na l N e tw o r k A ut ho r itie s
R eg io na l N e tw o r k A ut ho r it ie s
R e g io na l N e tw o r k A ut ho r it ie s
R e g io na l N etw o r k A ut ho r it ies
R e g io na l N e tw o r k A ut ho r itie s
R eg io na l N e tw o r k A ut ho r it ie s
E N -IS O 1 7 0 2 5
C a lib r a t io n ?
T esting ?
E C H a rm o n iza t io n
E N -IS O 1 7 0 2 5
IS O G uid e 4 3 ?
E N -IS O 1 7 0 2 5
T e sting
27
the individual network sites with the tiered approach, in contrast to the situation in
(a) above.
- The regional QA/QC unit, on the other hand, may have closer and more direct
contacts with the individual network sites than a QA/QC unit that operates
nationally, and therefore over a significantly larger region/geographical area. This
localised QA/QC unit may have positive effects on the quality of the results;
- There are nevertheless requirements within Article 3 of Directive 2008/50/EC for
the NRL to ensure the accuracies of all the measurements within its Member
State that are reported under the requirements of the directive (clearly within the
overall responsibility of the competent authority). This may be more difficult to
ensure since the individual network sites may not operate within the direct control
of the NRL. This is discussed further in Chapter 5.
- There is therefore a need for the NRL, as part of its responsibilities within that EU
Directive, to carry out audits and other QA/QC checks, so as to convince itself
and inform the competent authority that the regional or other QA/QC units are
providing continuously the required measurement uncertainties of all the results
and, other DQOs at the individual network sites, as required;
- The multi-step or tiered approach will usually result in worse measurement
uncertainties of the results at the individual measurement sites.
In the above cases, both for the single QA/QC unit at a national level, and for the
NRL at the centre of a tiered or regional QA/QC structure, it is useful to emphasize
that:
2.6.3 Dissemination for Methods that Use CRMs for Quality Control/Quality
Assurance: Ambient air
The EU Reference Methods for the ambient-air pollutants that are regulated under the
ambient-air directives, that utilize CRMs as part of their QA/QC procedures are those
analytical procedures that are used for the measurements of particulate-bound heavy
metals and PAHs, as outlined above. The networks that are operated to monitor these
species may, in principle, also be operated either through one laboratory at a national
28
level analysing all the PM filter samples, or through the use of a regional approach, in
a similar manner to that discussed in Section 6.1. However:
-
The methods for sampling the heavy metals and PAHs in the field are relatively
simple compared with the comprehensive QA/QC procedures required in the field
at the network sites for the gaseous species discussed in Section 6.1.Therefore it
may not be necessary to accredit these field sampling procedures for the cases of
heavy metals and PAHs, and this may reduce the requirements for a tiered
network.
There are significant advantages in analysing all the samples in one laboratory at a
national level from the perspective of the homogeneity of the results, and it is also
easier to audit the performance of that one laboratory and the accuracy of the
results obtained; However, this is not mandatory and is decided by the competent
authority in the Member State.
There may be cost benefits when all the samples are analysed in large batches;
The dissemination mechanisms and their related requirements are therefore clearly
different in the two cases:
a. For the case of one national laboratory carrying out all the analyses, a specific
dissemination mechanism is unnecessary, although there are requirements for this
laboratory to assess the methods whereby samples are taken in the field and
dispatched to the analytical laboratory to ensure the integrity of the samples. This
is more important in the case of PAHs sampling but is also relevant for heavy
metals. In addition, there are essential requirements to assess the quality of the
results obtained by the national analytical laboratory, and this should be carried out
through international comparisons, as discussed in Section 2.7 below.
b.
For the case of a tiered approach, where regional or other laboratories carry out
analyses within the same country, the NRL should carry out audits and other
QA/QC checks, in order to establish and demonstrate that the results conform to
the EU DQOs. This should be done using similar procedures to those described in
Section 2.7.
c. It is considered important in the case where there is a single laboratory carrying out
all analyses of heavy metals or PAHs in a country, that this is considered as the
NRL for those measurements, and therefore should obtain accreditation to the EN
ISO 17025 standard according procedures of the relevant CEN technical standard,
as required by the relevant EU directive. It is also necessary to take account of all
the results of international comparisons and audit checks within this accreditation.
d. There are as yet no specific requirements in directive 2004/107/EC, that these
heavy metals and PAHs measurements to be accredited to the EN ISO 17025
standard (unlike those in Directive 2008/50/EC). However, the CEN reference
methods require such accreditation - see for example EN 14902 clause 10.8)
e. When there is a tiered approach the NRL should obtain accreditation to cover all
the relevant technical activities it carries out, wherever practical. It is not
necessarily required, however, by the EU directives that the regional analytical
laboratories are accredited to the EN ISO 17025 standard. Then, however, it
should be emphasized that in these cases the NRL, as a representative of the
national competent authority, should take some responsibilities for the accuracies
of the results obtained by these, and this will require account to be taken of all
quality issues, including the results of audits and other QA/QC checks (see
Chapter 5).
29
30
discussed in Chapter 6. The results should be reviewed by all those involved with
these side by side comparisons, and by the relevant NRL, to ensure the data
quality of the results are as required by EU Directive 2008/50/EC.
Compare the calibration standards and the measurement capabilities NRLs making
measurements made in each EU Member State, so as to establish their validity,
comparability, and uniformity, and thereby to assist with the harmonization of
regulated European ambient air quality measurements;
Establish the accuracy and the measurement uncertainties of the results obtained
during these exercises and confirm that these meet the requirements of the
relevant directive;
Check the status and levels of implementation of the EU Air Quality directives;
31
The IEs organised by the ERLAP should, as described in the directives, involve all the
relevant NRLs designated for each of the pollutant species. Initially, these IEs
organised by ERLAP, which began in the early 1990s, were carried out for one
gaseous pollutant at a time. More recently, these have involved more than one
gaseous pollutant within each IE (see Chapter 6).
The World Health Organization (WHO) carries out similar IE activities, but with a view
to obtaining harmonised air quality data for health related studies, and this
organisation has integrated its programme within the WHO EURO Region, to include
public health institutes and other national institutes - especially from Central and
Eastern Europe, the Caucasus, and countries from Central Asia.
Recently, the IE activities of the JRC ERLAP have been merged with those of the
WHO, in order to:
A document has been prepared on this merged IE activity by the JRC, agreed with the
WHO and AQUILA (Reference 6), and these IE activities are now carried out jointly
between the JRC and the WHO. The AQUILA and WHO agreed document describes
further the procedure and data evaluation to be followed for IEs and can be
downloaded from AQUILAs website.
Until recently, the ERLAP IE activities concentrated on the gaseous pollutant species
of sulphur dioxide, the nitrogen oxides (particularly nitrogen dioxide), carbon
monoxide, and ozone. Examples are given in Chapter 6 of this document. Recently,
however, the IE activities of the JRC ERLAP have been extended to cover other
species such as the monitoring methods for benzene, for the 30 volatile organic
compounds specified in Annex X of Directive 2008/50/EC, for the analyses of heavy
metals, and the application of the PM10 and PM2.5 Reference Methods in the Member
States. These later comparisons have generally required somewhat different
methodologies than those described in Reference 6. They are all discussed in more
detail in Chapter 6 of this document, where examples of the results obtained are also
given.
2.7.3 International Comparisons carried out by the CCQM and Associated
Organisations
2.7.3.1
Background
32
These standards were prepared by one European Laboratory, and analysed by them
before and after they had been analysed by all the participants, in order to make
33
10
-10
VN
IIM
D
)
U
BA
(
N
PL
N
M
i
LN
E
KR
IS
S
-J
R
C
IE
S
IP
Q
FM
I
C
H
M
I
-20
C
ER
I/N
M
IJ
20
34
national standards are accurate and comparable before these calibrations are
disseminated, as discussed in Section 6.4. These comparisons were originally carried
out by sending the national SRPs back to NIST USA. However, with the proliferation of
these SRPs outside the USA, this became more difficult to manage. Therefore, it was
decided that the SRPs and similar Reference Photometers that are in located outside
of the USA (e.g. excluding those operated by the US EPA) and generally operated at a
national level, would have their calibrations checked, and if required adjusted, by
scientific staff within the International Bureau of Weights and Measures (BIPM), and
this would be carried out regularly through a series of international comparisons. The
first of these comparisons, organised under the auspices of the CCQM, was carried
out by BIPM, with SRPs from the different national laboratories sent to BIPM
sequentially between July 2003 and February 2005, using SRPs at BIPM and ones
provided by NIST USA as references. The results of this first comparison of Ozone
Reference Photometers carried out and reported by BIPM are shown in Figure 2.8. A
full report on this has been published (Reference 9).
(c)
Annex X of the above Directive requires Member States to monitor 30 volatile organic
hydrocarbon (VOC) species that are considered the dominant VOC species involved in
the photochemical production of ozone in the atmosphere near ground level.. These
species are generally present at ppb concentrations or below in ambient air. The
analyses of these VOC species are generally carried out by using gas
chromatography, with cryogenic pre-concentration of the atmospheric samples.
However, there are no specified performance characteristics and criteria for these
measurements.
It is practical to prepare calibration standards for these multi-component gas mixtures
using the gravimetric techniques outlined above in Section 5.2 above, and described in
more detail in the standards EN ISO 6142 and EN ISO 6143, at concentrations at
parts-per-billion (ppb) levels and below. There is, however, a requirement to
demonstrate the stability of these standards since they contain low concentration
reactive species.
The JRC ERLAP, in association with AQUILA, and the EURAMET Group, proposed
and carried out jointly an international comparison exercise using gas mixtures at
these low concentrations in high-pressure gas cylinders. Batches of gas mixtures,
were prepared both containing synthetic mixtures of these 30 VOCs at ~ 4ppb
concentrations, and also spiked ambient air mixtures generally at lower
concentrations. These standards were prepared by one European laboratory and
provided to the AQUILA and EURAMET participants, with concentrations unknown to
them. Stability trials were carried out during this EI, lasting about one year. This EI is
now completed, a report has been prepared by the JRC ERLAP (Reference 10), and
another has been prepared by EURAMET (Reference 11). Figure 2.9 shows the
EURAMET results obtained by the participating NMIs for some species.
Further comparison exercises covering specific volatile organic species were carried
out earlier by CCQM. These involved comparisons between NMIs worldwide of the
concentration of gas mixtures containing benzene, toluene, ethyl benzene, and the
xylenes, at concentrations of about 100ppb or 10ppb respectively, and multicomponent
mixtures of halogenated and other hydrocarbons at around 100ppb. The results have
been published (References 12,13, &14)
Figure 2.8: Summary of the Results of the Worldwide International Comparison
of Ozone Standard Reference Photometers (SRPs) and Similar Instruments -
35
36
Figure 2.9: Results of the EURAMET Comparison (886) for certain of the 30
Component Hydrocarbon Ozone Precursor mixtures specified in
Annex X of Directive 2008/50/EC, described in Section 2.7.3.2(c)
above
37
It covers all the quality management system requirements of the ISO 9001
standard that the organisation would need to implement to enable its
management practices, organisational structure, quality policy, process and
resource management etc. to be considered valid and defensible;
The requirements for the technical work to be carried out by the calibration or
testing laboratory are generally more comprehensive and rigorous than those
specified within EN 45001, which it effectively replaced, as discussed in Annex
2;
The EN ISO 17025 standard is compatible with the requirements of the current
internationally accepted method for the determination of measurement
38
This was emphasized when the GUM document, published by ISO, was also
transposed into a CEN standard Guide to the Expression of Uncertainty of
Measurement (ENV 13005 1999).
The adoption of the EN ISO 17025 standard as the definitive European (and
worldwide) quality assurance standard to underpin technical accreditations for
testing and calibration laboratories should therefore be considered as beneficial
and highly relevant to the 1996/62/EC Framework directives original requirements
for a European quality assurance standard as referred to in that directive. This
quality assurance standard EN ISO 17025 is now referred to specifically in the EU
Directive 2008/50/EC (see below).
Information on some of the more important scientific and technical requirements
within EN ISO 17025 and the GUM, which are specific and relevant to the scientific
and technical work of the NRLs, are given in Chapter 2 of this document.
(b) More recently, within a separate initiative, the European Commission developed
the new Directive within the Clean Air for Europe (CAFE) Programme, and this is
now published as Directive 2008/50/EC on ambient air quality and cleaner air for
Europe (Official Journal of the European Union L152 11.6.2008). In this 2008
Directive, in order to be consistent with the earlier Framework directive
1996/62/EC, there is an Article 3 with a very similar scope. However, this new
Directive has some additional text contained in Annex 1, which describes in more
detail than the 1996 directive, the methodology and context of ensuring the
accuracy of measurements and compliance with the data quality objectives. The
issues related to this requirement are discussed in Section 3.3 below.
3.3. General Quality Assurance and Quality Control Requirements for
39
This Section also defines WHICH of the laboratories of the Member States require
this accreditation. That is:
The laboratories that are designated to take part in the Community-wide
intercomparisons that are regulated in the Directive for the reference
methods referred to in Annex VI of that Directive. Other laboratories in the
Member States that do NOT take part in the Community-wide intercomparisons
for the reference methods do not require any such accreditation, although
recommendations for these are made below in this document (Section 3.4).
The above definition is also compatible with the implicit definition of an NRL within
Article 3 of the Directive as discussed above in Section 1.4.
(NOTE 6: The previously published 4th Daughter directive 2004/107/EC is not
covered by this 2008 EU Directive, and thus the exact provisions and requirements
listed here do not formally apply to this. However, it is anticipated that this may be
aligned in future).
(3) The definition as to which laboratories need to be accredited to the EN ISO 17025
standard leads on naturally to a detailed discussion of the SCOPE of the
accreditation required. In this context it should be understood that:
(a) It is often the case that more than one NRL will be designated in order to cover
the range of ambient air pollutants that are specified in Directive 2008/50/EC,
(and more within directive 2004/107/EC should this be relevant in future),
although this is not mandatory and the competent authority decides. Therefore,
it should be clear that in this circumstance it is necessary for the NRL to have a
scope of accreditation that relates only to the reference method(s) it has been
given responsibility for by the competent authority in the Member State.
(b) The text of Annex 1 Section C of the Directive states that, in addition to
restricting the requirements for accreditation to that laboratory that participates
in Community-wide intercomparisons it is necessary for the NRL to be
involved in one (or more) reference methods (within the Member State) as
specified in Annex VI. This is important in arriving at the requirements and
scope of their accreditation because:
o The reference methods specified in Annex VI cover a number of different
technical tasks. These include, as discussed in Section 1.4, the type
approval of the instrument before use, the initial installation, the set up and
the first calibrations and other tests required on the complete monitoring
system at the selected sites in the Member State, and the subsequent ongoing quality assurance, quality control, and maintenance of the complete
system during its field operations, so as to ensure the on-going required
accuracy of the results.
o When the NRL is involved in realizing and/or implementing one or more
of the technical activities listed above, for one or more regulated
pollutants, as specified in the CEN standards defining the reference
methods, and as given in Annex VI of the Directive, this range of
technical activities therefore MUST BE ACCREDITED according to the
requirements of the EN ISO 17025 standard. This should be considered
the minimum scope of the required accreditation. Bullet (iii) and Section
3.3 bullet (a) below gives strong recommendations on the OTHER
related TECHNICAL activities that should also be accredited
40
(4) The methodology for determining the measurement uncertainties of the methods
primarily ENV 13005:1999 and ISO GUM: 1993;
(5) The level of confidence of the measurement uncertainties that shall be used
95%, and how these shall be determined;
(6) The NRL and the competent authoritys obligations wherever the reference
measurement method is not used for the Directives purposes to carry out the
procedures specified in the EU Guidance on the Demonstration of Equivalence
(Reference 4) for non-reference Methods, or other relevant methods that are
specified by the national competent authority.
It is useful to emphasize at this point certain aspects concerning the laboratories that
require this accreditation, as a result of these definitions:
(i) This definition of which laboratories must be accredited is linked to their
participation in the Community-wide intercomparisons. This may not therefore
cover all laboratories within a given Member State that have national
responsibilities, due for example, to a particular federal structure, or due, for
example, to the fact that a different laboratory provides the nationally-traceable
calibration standards required.
In this case, the Directive limits (as a minimum) the requirements for
accreditation to the EN ISO 17025 standard to those laboratories that
participate in the Community-wide intercomparisons, as designated by the
relevant competent authority.
There are, however, some additional recommendations within this document to
cover these other laboratories that have national and possibly regional
responsibilities (see below Section 3.4).
(ii) It should not be possible for a laboratory to avoid the requirement for accreditation
to EN ISO 17025 by declining to participate in international comparisons that relate
to the relevant technical quality assurance activities of that NRL in the Member
State, since this would contravene the other requirements of Article 3 of the
directive, and EU treaty provisions on cooperation.
(iii) It may be argued from a reading of the Directive and its Annexes that the scope of
the accreditation should only cover and be limited to those technical activities that
are carried out directly within the Community-wide intercomparisons. However,
this is not practical since it is not possible to accredit a specific activity that may
take place for a short time once in every three to five years. In addition, the
activities carried out during these Community-wide intercomparisons should
generally be considered to be a sub-set of the activities that are carried out by the
NRLs during their role within the Member State, and thus the accreditation as
indicated above, should have a scope that covers all the technical qualityassurance activities that the specific NRL carries out at a national level
within the CEN reference method(s) for which it has responsibility.
In addition, importantly, the results obtained during these Community-wide
intercomparisons must be made available during the assessments of the
accreditations to EN ISO 17025, in order to provide independent evidence that the
results obtained by the NRL are valid within their stated uncertainties. Any nonconformances obtained as a result with the measurement uncertainties stated in
their scope of accreditation that are apparent as a result of these international
comparisons, must be investigated and rectified.
41
(iv) As discussed above, a number of CEN reference methods cover separable and
different experimental activities, including type approval for automated method
before its use, and the regular ongoing QA/QC activities carried out whilst the
automated method is deployed in the field. There are clearly some common
features of these but in detail they are actually different activities. Therefore,
where a given laboratory is involved in one (or more) or these technical
activities, the scope of the accreditation shall cover this fully.
There is no specific requirement for a scope of accreditation that covers the
other activities within the CEN reference method that the NRL is not involved
with directly in a technical manner.
However, the NRL has certain responsibilities within these other activities and for
other technical activities outside of the scope of the CEN reference methods, as
outlined below in this document.
(v) The primary activities of the NRL that require accreditation to the EN ISO 17025
standard are those stated above. These may be considered to represent the
minimum scope of the required accreditation. This scope, and the related
detailed technical procedures and other quality management documents that
embrace all the quality-assurance and quality-control activities involved,
should be comprehensive and cover all of the national activities carried out
technically by the specific NRL to ensure the quality and accuracy of the
measurement results supplied to the Commission.
It is considered that this should also include and describe the relevant
Community-wide intercomparisons as these provide an essential means of
assessing, in an audit manner, the validity of the accredited activities carried out
at a national level by the NRL. These comparisons therefore should be considered,
as discussed above, as an integral part of the national QA/QC activities using
the reference method.
(vi) The accreditation requirements within the EN ISO 17025 standard cover both
calibration and testing activities for a range of technical activities, with (almost)
equal requirements (indeed most European accreditation bodies treat these now
as equally stringent). When defining whether the requirements discussed here are
for a testing or a calibration accreditation of the technical quality assurance
activities carried out by NRLs, within the ongoing QA/QC requirements of the CEN
reference methods, it should be noted that most should be considered as testing
activities (e.g. linearity and cross sensitivity tests on the complete analytical
system). However, some of the technical activities carried out may be considered
calibrations (e.g. the certification of a gas mixture used in the field with the
analytical system against a nationally-traceable calibration standard, in order to
confirm that the system remains within specified calibration responses as given in
the CEN reference method). Then wherever these gas mixtures are used to
calibrate/recalibrate the reference method this should be considered as a
calibration activity and be accredited accordingly.
In addition, in the past, it was recognised that testing accreditations did not
necessarily require demonstrable measurement uncertainty statements assigned
to the results. This is clearly not the case for ambient air quality measurements.
It is recommended, therefore, that the accreditation is normally that for
testing. However, the laboratory concerned, in consultation with the relevant
national accreditation body, should consider the issue of whether a
calibration or testing accreditation is appropriate in part or for the whole. In
certain cases where there is a strong dependence on the accuracy of the
calibrated artefact used in the field on the quality of the results, then this
should be emphasized in the accreditation. However, in all cases there is
clearly a requirement to emphasize and make explicit the measurement
42
Therefore, it is strongly recommended and highly desirable, that all the technical
activities carried out by the NRL, that ARE associated with the above two activities,
and that are clearly additional (but related) to those discussed in Section 3.3 bullet
(iv) should also be included in their scope, and the technical procedures
covered by the accreditation of the NRL to the EN ISO 17025 standard,
wherever applicable and part of their responsibilities.
(b) Furthermore, where the NRL is not involved in all of the activities specified in the
CEN reference method - for example, if the NRL is not involved in those tasks
associated with type-approval of the method, or in its initial installation and
commissioning at a field site, failure to implement these activities correctly will
mean that the accuracy of the measurements may be compromised. Therefore, in
these circumstances, it can be argued that the NRL has a level of responsibility for
ensuring that all such activities have been carried out appropriately, with the
required measurement uncertainties, and within the required conditions (unless
this has been delegated to another organisation by the competent authority).
Thus it is not necessary for a given NRL to carry out the type approval and the
other tasks itself, but it is necessary for the NRL to convince itself that these
activities have been carried out in order to perform its function appropriately, and
to appropriate requirements, unless the competent authority has taken this
responsibility.
43
.
(NOTE 7: The 2008 Directive also makes recommendations that the competent
authorities accept the type-approval testing and results produced by laboratories in
other Member States, provided these have been carried out by laboratories
accredited appropriately to EN ISO 17025 for carrying out such testing (see 2008
Directive Annex VI section E), and the appropriateness or otherwise of the typeapproval this should be considered by the relevant NRL, and advice be supplied if
required to the competent authority).
(c) Laboratories other than NRLs may operate, as outlined above, at a national level
but they may not participate in Community-wide intercomparisons and therefore
are not required by the Directive to gain an accreditation as discussed above to the
EN ISO 17025 standard.
However, In addition, Annex 1 Section C states that institutions operating
networks and individual stations shall have established quality assurance and
quality control systems to assure the accuracy of the measuring devices. This is
intended to be institutions at a regional, local, or more limited level that are not
required by the Directive to gain accreditation to EN ISO 17025.
In these circumstances, it should be argued that the NRL has a level of
responsibility for ensuring the accuracy of measurements and requirements for
measurement traceability, although no accreditation exists for these, unless
otherwise delegated to another organisation by the competent authority. It is
recommended, nevertheless, that such organisations:
gain the appropriate accreditation, where this is practical, or if not they should
follow all the requirements of the EN ISO 17025 standard that are possible;
It is important that the NRL play a significant role in these above activities, in
addition to their other responsibilities.
(d) There are further requirements in article 3 of the 2008 Directive that the NRL
ensures the accuracy of measurements covering pollutants that are regulated in
the 2008 Directive (and the 4th daughter directive). There are clearly a number of
mechanisms for demonstrating this, and some of these are discussed below as
specific examples. One of these relates to the requirement for traceability of all
measurements to recognised national or international standards, following the
requirements set out in Section 5.6.2 of the EN ISO 17025 standard. This
requirement is discussed in more detail in Chapter 2.
(e) The appropriate accreditation of the NRLs to the ISO/IEC quality-assurance
standard, with the scope as discussed above, must be in place by the end of the
year 2010.
44
Instruments or analysers that are selected for use in order to meet the regulatory
requirements in relevant EU directives (see below);
In addition, it can also be seen from this definition that the type approval procedure
and its associated tests, as applied to commercially manufactured instruments, may
be implemented only once during the lifetime of this ONE TYPE of instrument
throughout its entire time frame of manufacture and operations, which is
generally five years or more. Then if only this one-off type approval procedure is
carried out generally at the beginning of the lifetime of one type of analyser, there is
therefore no guarantee that modifications may have occurred during this lifetime that
may affect the performance of that type of instrument with respect to the EU directives
data quality objectives. Therefore CEN has developed standards (see Section 4.3
below) that provide ongoing requirements and control of the analyser performance
throughout its lifetime.
The concept of instrument certification, or product certification, builds on the above
type approval, and its related tests, but it imposes additional on-going
requirements on the manufacturer of this instrument, on the designated body
and on the end-user, before and after these type-approval tests, to address the issue
of the possible one-off nature of these type-approval tests. The type-approval
requirements are summarized in Section 4.2 below. The additional requirements,
resulting from the need for instrument certification or for product certification have also
been incorporated into CEN standards, and these are discussed below in Section 4.3.
The drivers for the type approval of instruments used to monitor ambient air quality for
the purposes of meeting the regulatory requirements of the EU directives, arose
originally from the EU Framework, where article 3 states:
45
It contains the same text as in Article 3 of the above with the same implications;
There are further requirements for a range of different tests if the Reference
Methods are not used (Section B Annex VI);
46
The EN standard 15852 for the measurement of total gaseous mercury, which is
based on a semi-continuous method;
4.2.2 Summary of Type-approval Requirements within the CEN Standards
The type-approval tests are used to evaluate a set of performance characteristics
determined under a prescribed series of tests. Procedures are defined for the
determination of the actual values of the performance characteristics for at least two
identical types of analysers in a laboratory and in the field. The standard specifies a
comprehensive and rigorous set of tests. The evaluation for type approval of an
analyser is based on the conformance of the analyser with all the individual
performance criteria that are specified in the standard and determined using these
tests. It also requires the calculation of the overall expanded uncertainty of the
measurement result based on the numerical values of all the relevant tested
performance characteristics, and this is compared with the relevant prescribed
maximum uncertainty specified in the Directives DQOs (e.g. 15% at 95% confidence
level for NO2 at the hourly limit value). The tests and calculations carried out are:
Laboratory tests:
- Short-term drift tests at zero and span concentrations (12 hrs);
- Response times rising and falling;
- Repeatability at zero concentrations and at the highest numerical limit or target
value (where appropriate);
- Lack of fit (residuals from the linear regression function including the zero value);
- Sensitivity to sample gas pressure and sample temperature;
- Sensitivity to electrical voltage;
- Sensitivity to the temperature surrounding the analyser;
- Effects of interferences from other substances present in the atmospheric samples;
- Effect of fluctuating concentrations (averaging test);
- Converter efficiency where applicable (NOx)
- Carry over where applicable (benzene)
- Differences between sample and span inputs, where applicable.
Field tests (3 months):
- Long-term reproducibility between two analysers during the field measurements;
- Long-term drifts at zero and span levels;
- Period of unattended operation, and fraction of total time the analyser is available
for measurements.
Overall uncertainty of measurement results:
- Determined by combining the type approval results obtained from the relevant tests
above, expressed at a 95% confidence level.
47
and have the required on-going quality assurance and quality control procedures
applied, as specified in the relevant CEN standards. These CEN standards also
contain rigorous laboratory and field tests to establish the specified numerical values
for all the relevant performance characteristics that contribute to the overall
measurement uncertainty of the method. They are mandatory to be implemented in the
EU Member States.
(NOTE 9: This situation is in contrast to the technical requirements for industrial stack
emission analysers, where no mandatory standards were available at a European level
before the publication of standard EN 15267-Part 3, where similar technical
information and test requirements are contained in this one standard for all pollutant
species see below).
There are, however, additional requirements for other standards at a European
level that do not provide the technical and the testing requirements of measurement
methods, but provide instead an acceptable formal management and organisational
structure under which these technical CEN standards can be operated. These
requirements are covered by two CEN standards that have recently been published.
These are EN 15267 Part 1 and EN 15267 part 2. A summary of the scope and
objectives of these two standards is given below:
4.3.2
4.3.2.1
EN 15267 Part 1:
General principles
The type approval of Automated Measurement Systems (AMSs), and their subsequent
certification, is broader than that specified in Directive 2008/50/EC, and supports the
requirements of other EU directives, particularly those related to the emissions of air
pollutants to atmosphere from certain industrial plants that are regulated by EU
directives.
The responsibility for ensuring that AMSs that monitor ambient air quality under
Directive 2008/50/EC in a given Member State is clearly defined as the national
competent authority in the Member State. However, no such explicit requirements are
specified in the relevant EU directives for approving AMSs for industrial sources.
Instead, the competent authorities in some Member States have set their own
arrangements in place.
In certain Member States the competent authorities have designated the type approval
or certification of AMSs that monitor both ambient air quality and industrial emissions
to one or more organisations that are accredited to standard EN 45011:1998 (General
requirements for bodies operating product certification schemes identical to ISO/IEC
Guide 65:1996) by a national accreditation body. However, In some cases the
competent body itself is not accredited by an external body to EN45011, whereas in
other cases it may be. In some States the designated organisations may be public
bodies, in others private. These different approaches have been developed over a
number of years, and reflect the different legislative and administrative arrangements
that exist in different Member States (see also Section 4.5 below).
European standard EN 45011 specifies the general criteria that the certification body
shall follow if it is to be recognised as at a national or European level as competent
and reliable. This is one step towards mutual recognition, as outlined further below.
48
4.3.2.2
49
The standard generally follows the format and requirements of EN ISO 9001:2000.
However, it does not preclude the use of other quality management systems that are
comparable in objectives. Where no such quality management system exists there are
requirements for enhanced auditing of the manufacturer by skilled personnel with the
control of the certification body. In addition, there will be continuing evidence available
of the performance of the AMSs from their use in field measurements.
4.4
The certification procedures outlined above, and covered in the standards EN 15267
parts 1 & 2, together with the relevant CEN standards discussed in Section 2.1 above,
require, in principle, that in each Member State a certification body, and one or more
type-approval and performance testing organisations are appointed and operate,
together with the related activities that are needed, and with guidance from the
relevant stakeholders. It is clearly the responsibility of the competent authority in the
Member State:
-
to agree this and to establish suitable bodies within the Member State or
elsewhere;
or to propose alternative methods that it can justify;
and/or to agree that it will accept the appropriate test reports from other
appropriate test laboratories in other Member States as specified in the Directive
008/50/EC Annex VI Section E
Within this overall framework the NRL might or might not serve as a recognised testing
laboratory or the certification body. However, these roles and responsibilities should
not be considered as integral and implicit parts of the normal responsibilities of the
NRLs. Instead it may be considered in a Member State that the NRL, in its role of
ensuring the accuracy of the results, should be to formally or informally act as an
technical assessor or technical reviewer of these activities, in order for example to:
Review the validity of the testing programme and the results obtained so as to
ensure that the technical quality of the Reference Method being implemented
conforms to the requirements of the Directives DQOs;
Review the results obtained through QA/QC procedures in the field to check that
the Reference Method continues to operate in long-term use;
Exchange information through AQUILA, and related mechanisms, about the
testing, type-approval and certification mechanisms, so as to improve the existing
CEN standards through future revisions;
Thereby improving the confidence, the transparency, and the mutual acceptance of
the test results.
An additional role for the NRL may be considered, where appropriate, as being
involved in the testing of methods other than the Reference Methods for equivalence,
or in overseeing or reviewing the technical data from such testing, in order to ensure
its validity. This should apply especially, but not exclusively, to PM equivalence testing.
An NRL should therefore also be willing to collate and exchange the technical
information and the data obtained from equivalence testing with other NRLs through
AQUILA, in order to achieve a more harmonised and rigorous approach across Europe
in future. It may also become more important for other pollutant monitoring techniques
including those related to passive sampling.
50
4.5
As outlined above, European standard EN 45011 specifies the general criteria that a
certification body that operates a product certification scheme must follow should it is
seek to be recognised at a European or wider international level. To support this,
Document EA6/01 published by the International Accreditation Forum (IAF) provides
guidance on the application of EN 45011, with the purpose of harmonising
internationally the application of EN 45011 by accreditation bodies. This is one
important step towards the mutual recognition between certification bodies in different
countries through the IAF Multilateral Agreement.
The preparation and publication of the two European standards EN 15267 parts 1 & 2
discussed above by CEN Technical Committee 264 are also intended to facilitate this
in the field of ambient air quality and industrial emission monitoring, by providing
clarification and technical interpretation of the relevant generic standards. The aim
therefore is the mutual recognition, where practical, of the certifications and testing for
type approval or product conformance of AMSs carried out on behalf of recognised
bodies, so that there is no absolute requirement for each Member State to carry out
this certification process should it choose to accept the results from another Member
State.
This objective is also intended in Directive 2008/50/EC Annex VI section E:
In carrying out the type approval to demonstrate that the instrument meets the
performance requirements of the reference method, designated competent authorities
and bodies shall accept test reports issued in other Member States by laboratories
accredited to EN ISO 17025 for carrying out such testing
This mutual recognition of testing reports that conform to the EN ISO 17025 standard
requirements for the specified tests is already in place through international
agreements, either formal or informal, within certain Member States - for example
between UBA Germany (where type approval is carried out by TV and other
laboratories) and the UK MCERTS certification scheme established by the
Environment Agency of England and Wales. Other countries (e.g. France and Italy
now have in place, or are planning to establish, their own type-approval schemes for
these types of AMSs (and industrial-emission monitors), and there is a wish for mutual
recognition in these countries, and in certain others also. This is one important step
towards a more pan-European certification/type approval scheme in the future, to
avoid unnecessary and expensive duplication of the testing work required.
There will need, however, to be very considerable improvements to the harmonisation
of the testing, and more transparency of the tests themselves and the results obtained
from these in practice, if this is to be achieved widely across Europe. The European
standards discussed above, the overarching management and organisational
standards, EN 15267 parts 1 & 2, and the relevant Reference Methods, are intended
to provide the mechanisms for bringing this about, probably in the longer term.
51
5.1 Introduction
The tasks specified for the Competent Authority/National Reference Laboratory in the
Directive 2008/50/EC on Ambient Air Quality and Cleaner Air for Europe have been
summarized in Section 1.4 of this document and the roles of NRLs summarized
elsewhere. This Chapter covers in more detail the following specific technical
requirements at a national level:
Directive 2008/50/EC Article 3 of point (c.):
Member States shall designate at the appropriate levels the competent authorities and
bodies responsible for the following:
Where relevant, the competent authorities and bodies shall comply with Section C of
Annex I.
Directive 2008/50/EC Annex 1 Section C:
1. To ensure accuracy of measurements and compliance with the data quality
objectives laid down in Section A, the appropriate competent authorities and
bodies designated pursuant to Article 3 shall ensure the following:
2. All reported data under Article 27 shall be deemed to be valid except data flagged
as provisional.
As discussed previously in this report, it is generally accepted that the Competent
Authority of a given Member State is designated at the political level, whereas national
laboratories or National Reference Laboratories are designated at the technical level,
52
this distinction is not clear in the Directive and hence, on a number of occasions it is not
clear whether the responsibilities for certain of the activities discussed in this Chapter
belong to the competent authority or to the National Reference Laboratory. Thus these
two terms may be used in combination in this Chapter.
The Article 3 of Directive 2008/50/EC concerned with ensuring the accuracy of
measurements must be interpreted in the context of the required Data Quality Objectives
(DQO) listed in Annex 1 of that Directive. These specify uncertainty, minimum data
capture, minimum time coverage (where appropriate) for fixed measurements, the
requirements for indicative measurements, modelling and objective assessment, and for
each pollutant covered by the Directive. This Chapter will address fixed and indicative
measurements.
These DQO requirements apply to individual analysers/samplers at individual stations. In
order for these measurements to constitute a compliant overall assessment of air quality
in the Member State, the requirements for the appropriate numbers of monitoring points
in Zones and Agglomerations (Annex V and Annex IX), their locations and macro and
micro siting (Annex III and Annex VIII) must also be met. These requirements are
complex, with the need to maintain specific ratios of roadside and background site for
nitrogen dioxide, particulate matter, benzene and carbon monoxide:, specific ratios of
PM10 to PM2.5 sites, PM2.5 sites to assess exposure reduction, sites to assess
compliance with critical levels for vegetation and separate requirements for ozone
precursors monitoring. It would therefore be beneficial if an EU-level Working Group
were to be formed to provide additional guidance to Member States on the requirements
for the appropriate numbers of monitoring stations. At the least, this working group
should examine how these requirements have been interpreted in the different counties
and make recommendations. In the absence of this, this should be the subject of an
AQUILA document in future additional to this current document.
As noted previously, the Reference Methods of measurement for the various pollutants
are defined as the relevant CEN Standard Methods:
SO2
NO.and NO2
PM10
PM2.5
Benzene
CO
Ozone
Pb, Cd, As, Ni in PM10
PAH (benzo[a]pyrene)
EN 14212:2005
EN 14211:2005
EN 12341:1999
EN 14907:2005
EN 14662:2005
EN 14626:2005
EN 14625:2005
EN 14902:2005
EN 15549:2008
Of the above, as is well known, the last two standards (apart from requirements for the
monitoring of the heavy metal lead) are not covered by Directive 2008/50/EC. However,
this AQUILA document and this discussion still embrace these, where applicable, since it
is considered that this is an appropriate method for ensuring the DQOs for those
pollutants also.
The CEN standard methods for gaseous pollutants provide details of the reference
method of measurement for each pollutant, specific analyser tests to be performed and
requirements for the on-going regular quality control and quality assurance (QA/QC)
activities for the analysers and associated systems for sampling, data collection and
data transmission. In addition, these standards provide a methodology for uncertainty
evaluation to enable the overall measurement uncertainty of the method to be directly
compared with the specified DQOs.
53
This Chapter outlines some of the general principles of QA/QC, and the detailed CEN
QA/QC requirements - many of which will need to be newly introduced into
measurement networks - and provide guidelines and recommendations to NRLs on the
implementation of this QA/QC.
(NOTE 11: It will also be useful if the experience of network operators in the practical
implementation of these standards is fed back to the CEN working groups so that this
experience can be used when the current CEN standards are revised).
5.2
For ambient air quality monitoring networks, quality assurance activities should
include:
Network design;
Instrument selection;
Instrument operations;
Operator training;
Instrument calibration;
In principle, where all of the quality assurance activities are undertaken correctly, are
fully in compliance with the relevant CEN standard, and compliant with the detailed
local operating procedures at all times, then there is an expectation that the
measurements will fulfil all the requirements of the EU directive(s) without further
checking. However, in practice in large and complex networks, often with many
54
hundreds of analysers and ancillary equipment and many site operators, this is may
not always be the case. Hence, there is a need to control the quality of data by careful
data management and rigorous checking, preferably by very experienced personnel.
As an example, analyser faults must be identified and addressed quickly in order to
fulfil the Data Quality Objective for >90% data capture.
In addition, prior to submission of data to the EU, all suspect data must be identified
and investigated in order to fulfil the Directives requirement that all data reported
under Article 27 shall be deemed to be valid except data flagged as provisional
(Annex I, C). In addition to these technical requirements there is the need to ensure
that the data are reported correctly these tasks all require different skills.
Laboratories familiar with the requirements of the EN ISO17025 standard may be well
aware of the need to review quality procedures and activities regularly in order to
identify any generic issues that need to be addressed to ensure quality is maintained
and improved in future. We recommend that all network operators undertake a
regular review of all QA/QC activities, that the findings documented, and that
any necessary actions or improvements implemented.
The CEN standards set a number of operational condition checks that need to be
performed for each site and analyser deployed. These are summarized as follows:
55
These parameters need to be reviewed in order to ensure that all the analysers at all
the sites are operating within the operating conditions for which they have been type
approved. The consequence for the end-user is the possibility to use values of certain
performance characteristics obtained during the type approval test for his own
uncertainty calculation (i.e. linearity, sensitivity coefficient). This requires also to
determine the required frequency of certain maintenance actions, which may depend
on the pollutant concentrations at the particular site, or other factors. It will be very
useful to continuously monitor the temperature of the container used for the analysers
to ensure that the temperature control system (air conditioning system) maintains the
correct operating conditions for the analysers.
5.3.2.2
The CEN standards also specify a number of on-going QA/QC checks that need to be
undertaken whilst any gaseous analyser is in operation at a monitoring site:
The National Reference Laboratory should ensure, wherever relevant, that procedures
are in place for these on-going QA/QC checks and that results are documented and
any necessary remedial actions taken. When analysers are found to be non-compliant
with any of these checks then the data and site/analyser records must be examined to
determine if data need to be rejected of corrected.
5.3.3 Data Handling
The Competent Authority/National Reference Laboratory has the responsibility for
producing valid data. All reported data under Article 27 shall be deemed to be valid
except data flagged as provisional. Hence, all faulty data, and zero/span checks,
calibrations etc, must be removed from the dataset.
For NOx analysers, there is a specific requirement to correct data where converter
efficiencies are measured to be between 95% and 100% (note: it is possible with a
poorly calibrated analyser to determine more than 100% but this should be rectified
during QA/QC). If the converter efficiency of the analyser is below 95%, the data must
be rejected. Data for gaseous pollutants needs to be reported at conditions of standard
temperature and pressure, whereas PM10 and PM2.5 data need to be reported at
ambient temperature and pressure conditions at the date of measurement. (Annex VI,
C.)
Currently an EC Data Exchange Group (DEG) is in place, and any specific
requirements from this group, such as procedures for dealing with negative numerical
data, data below the detection limit of the analyser, and number of decimal places
56
required for data values, all need to be complied with before the data are submitted to
the Commission. The DEG will also specify what meta-data are required for each
monitoring station. The Competent Authority/National Reference Laboratory therefore
needs to ensure that these data are collected, accurate, and up-to-date, and reported
with the relevant measurement data.
(NOTE 12: The AQUILA Group is currently contributing to the future specifications of
these requirements, and the outcomes of this will be the subject of a separate report.)
5.3.4
The CEN standards provide a specific and comprehensive methodology for calculation
of the overall expanded uncertainty of measurement for direct comparisons with the
Directives DQOs. The CEN standard requires that this is calculated annually and it is
likely that the DEG will require that this to be calculated individually from all the QA/QC
data for each analyser. This should then be reported annually to the Commission
along with the corresponding annual data set.
5.4
5.5
57
The Competent Authority/National Reference Laboratory may also wish to audit these
other institutions to ensure that the written procedures are being implemented fully. In
addition, comparisons between these organisations and the Competent
Authority/National Reference Laboratory should be organised, where practical using
similar methodologies to those of the EU wide comparisons. In several Member States
these procedures are already in place. In some Member States, annual technical
meetings and/or calibration workshops are arranged as one alternative.
The JRC have also organised intermittently EU wide audits of selected monitoring
sites in each Member State for example a programme for particle measurements is
currently underway, and this is discussed in Chapter 6. Member States may also agree
to undertake mutual audits of each others sites, either bilaterally or within groups, to
assist in the harmonisation of the results and to facilitate exchanges of knowledge and
experience. This is an issue that should be discussed within the responsibilities of the
AQUILA Group.
5.6
5.7
Demonstration of Equivalence
58
Many of these issues are dealt with in the revision of the EC Guidance on
Equivalence published in June 2009 (Reference 4), and others will be resolved as
experience grows - with experimental activities carried out in future for demonstrating
equivalence and using equivalent methods. The AQUILA Group provides a suitable
vehicle for collecting such experiences and providing inputs to any further revisions of
the EU Guidance document and/or the related CEN standard(s).
5.9
59
60
The reference value (true value) of the known concentration artefact and its
measurement uncertainty: However, in the absence of a commonly accepted
value the average or median of the participants results may be used as the
reference. For an ILC at a national level it is recommended that the reference value
and its uncertainty are established by the responsible NRL with traceability to
recognised national standards.
The quality requirement: Generally, the standard deviation (or multiples of it) is
often used to define a quality requirement. The disadvantage of such an approach
is that this data quality criterion is dependent on the specific experiment carried out
and also on the proficiency of the participants and their calibration methods.
For air quality measurements that are required to be carried out under the
European Directives data quality objectives (DQOs), the measurement uncertainty
requirements are defined and have to be met in routine/regular measurements.
Therefore, the data quality requirements in the relevant ILC must also be linked to
the EU-DQO. This can be illustrated by the following example:
In Germany, private laboratories may be employed when these are notified by the
German states ('Lnder') to perform air pollution measurements according to the
German Federal Air Pollution Control Act. In this framework of QA/QC measures,
these laboratories in Germany must participate in inter-laboratory comparisons for
sulphur dioxide, nitrogen dioxide, and benzene every three years. These round robin
tests are evaluated according to the so-called z-score method based on ISO Guide 431 as follows:
z-score z =
x:
X:
:
x X
value of participant
reference ('true') value
required precision
|z| 2
2 < |z| < 3
|z| 3
('good')
('questionable')
('poor')
The reference value (X) is generally determined and provided by the relevant national
reference laboratory. The required precision () is based on the data quality objectives
of the relevant EU directive.
However, inter-laboratory comparisons take place within more or less ideal conditions,
since compared with field conditions, only some components of all the sources of the
overall measurement uncertainty are included. Examples of uncertainty sources not
considered in inter-laboratory comparisons may be:
Because of these considerations it was decided that the expanded uncertainty of the
participants values Ulab shall be below 50% of the data quality objectives, which
means:
61
2
2
U lab
+ U ref
= 2
with
SO2: 5 g/m
NO2: 4 g/m
benzene: 0.5 g/m
Since 2004 this evaluation scheme has been routinely used and widely accepted for
the evaluation of inter-laboratory comparisons for private laboratories in Germany.
5.9.4 Links to ILCs at a European Level
Where ILCs are organised at a national level in the MSs, it is important that they are
carried out rigorously, and are evaluated in a comparable way, as outlined above.
These procedures should in turn be linked and be consistent with those carried out at
a European level (see Chapter 6).
62
networks need to be replaced with type-approved analysers within five years of the
implementation of Directive 2008/50/EC
3. All of the CEN QA/QC procedures must be introduced into the monitoring networks
4. Data uncertainties must be known for every analyser, every year. It should be
calculated using the CEN methodology. Development of semi-automated systems
of performance evaluation and calculation of uncertainty based on known
procedures, calibration results etc., are recommended for effective processing and
report generation.
5. The appropriate accreditation for all the relevant technical activities, as discussed
in Chapter 4 of this document must be obtained, that conforms to the requirements
of the EN ISO 17025 standard;
6. The NRL must participate in the appropriate EU Community-wide quality
assurance programme(s).
7. The technical procedures and all the quality issues must be reviewed on a regular
basis in order to adapt and improve these.
8. Ensure that satisfactory documented quality-assurance procedures are in place for
all institutions operating networks and individual stations within the Member State;
9. Define and carry out appropriate technical audits of the measurements at all the
network stations, on an intermittent basis, with a frequency that is defined by the
quality of the results obtained;
10. Where equivalent analysers are used, ensure that the correct procedures have
been followed for the demonstration of equivalence and that a report has been
supplied to the Commission for approval;
11. Ensure that all data reported to the Commission comply with all the Directives
requirements and the recommendations of the Data Exchange Group, unless the
competent authority delegates this to another body.
63
pollutants, organics (BTX and VOC precursors), particulate matter, and its
constituents.
6.2 Examples for Inorganic Gaseous Compounds: Ozone, Carbon
54.0
52.0
SO 2 (nmol/mol)
50.0
48.0
46.0
44.0
42.0
40.0
38.0
36.0
A
Laboratory
Figure 6.1 Example of one run SO2 at a concentration level of about 45 ppb
(~ 130 g/m3) during an IE in 2007 with 15 participating laboratories
10.0
CO (mol/mol)
9.5
9.0
8.5
8.0
7.5
7.0
6.5
6.0
A
Laboratory
64
66.0
O 3 (nmol/mol)
64.0
62.0
60.0
58.0
56.0
54.0
52.0
A
Laboratory
Figure 6.3 Example of one test with ozone at a concentration level of about 60
ppb (~ 120 g/m3) during an IE in 2007 with 15 participating laboratories
Nitroge n dioxide conce ntration le ve l 10
21.0
NO2 (nmol/mol)
19.0
17.0
15.0
13.0
11.0
9.0
A
Laboratory
Figure 6.4 Example of one test with NO2 at a concentration level of about 13 ppb
(~ 30 g/m3) during an IE in 2007 with 15 participating laboratories
6.3
6.3.1 BTEX
A first EC-JRC aromatic species (Benzene, toluene, ethyl benzenes, xylenes - BTEX)
inter-laboratory comparison using automatic analysers took place at the JRC Ispra
facility in 2005 [EUR 22523 EN], and the second BTEX comparison took place in 2008.
During the second BTEX IE for benzene, the average reproducibility standard
deviation for the exercise (about 18 %) is in contrast with the relatively low repeatability
standard deviation (1.4 %). This lack of robustness ( ~ 17) is an indication of the need
65
for traceability in the calibration process and also for improvements in the instruments
linearities.
Benzene reproducibility standard deviation values of about 15 % at the limit value (5
g/m3) are very close to the reproducibility value obtained in the last intercomparison
(12.5%). Considering the higher associated uncertainty due to the limited number of
participating laboratories, no changes in the performance of the method can be
derived from this intercomparison with respect to the previous exercise.
Z-score is revealed as an appropriate criterion for the evaluation of laboratory
performance when a limited number of participating laboratories are involved.
Nevertheless, the calculation given in Document N37 (Annex 1) for the reproducibility
standard deviation for proficiency assessment seems to be very restrictive for the
method. In spite of that, almost half of the participants passed the Z-score test given in
N37 for benzene measurement proficiency testing.
Further harmonisation actions need to be implemented in order to obtain
reproducibility values, which can satisfy the N37 criterion for benzene. Otherwise, the
proposed N37 standard deviation may need to be reconsidered in order to fit into the
methods performance limits.
Table 6.1.- Average repeatability, reproducibility and gamma values of the
exercise - gamma values higher than two indicate a lack of robustness.
Benzene
Toluene
ethyl-benzene
mp-xylene
o-xylene
Repeatability, %
1.4
1.8
2.2
4.2
3.1
Reproducibility, %
17.8
10.0
9.7
8.0
16.5
17.2
7.1
6.1
2.1
6.7
66
The heaviest hydrocarbons (123, 124 and 135 trimethyl benzene, m,p-, o- xylene,
ethyl-benzene and benzene) and those more reactive and lighter (like 1,3 butadiene,
isoprene, acetylene and ethene) were analysed with more difficulty in comparison to
the rest of the compounds.
ethane
1,2,3-trimethyl benzene
1,2,4-trimethyl benzene
1,3,5-trimethyl benzene
o-xylene
20
ethene
propane
propene
15
iso-butane
10
m+p-xylene
ethyl-benzene
n-butane
acetylene
toluene
trans-2-butene
n-octane
1-butene
2,2,4-trimethyl pentane
cis-2-butene
benzene
2-methyl butane
n-heptane
n-pentane
isoprene
n-hexane
2-methyl pentane
1,3-butadiene
trans-2pentene
1-pentene
Figure 6.5.- Median values for the associated expanded uncertainties of the
reported concentrations of the synthetic and ambient air mixture
25.0
Synthetic mixture
Ambient air mixture
20.0
Median bias, %
15.0
10.0
5.0
0.0
67
1,2,3-trimethyl benzene
1,2,4-trimethyl benzene
o-xylene
1,3,5-trimethyl benzene
m+p-xylene
toluene
ethyl-benzene
n-octane
benzene
2,2,4-trimethyl pentane
isoprene
n-heptane
n-hexane
1-pentene
2-methyl pentane
trans-2pentene
n-pentane
1,3-butadiene
2-methyl butane
1-butene
cis-2-butene
trans-2-butene
n-butane
acetylene
propene
iso-butane
ethene
propane
ethane
-5.0
6.4
68
The results of the IE showed that for 77 % of the analyses, the uncertainty of
measurements estimated by participants was consistent with the differences between
the participant results and the reference values of the test samples. On average,
participants claimed uncertainties that were consistent with the DQOs: about 10 % for
Pb and between 15% and 20 % for As, Cd and Ni. The discrepancies were mainly
produced either by some participants underestimating their uncertainty of
measurements or by the explosions, a wrong dilution factor, contamination, high limit
of detection, or the interference mentioned above.
Once these special events are discarded, the reproducibility for all samples and
participants show values between 41% and 54 %. These figures are consistent with
the DQOs if one takes into consideration that reproducibility should be compared to 2
of the DQOs. The repeatability remains between 5% and 12 % without much
difference according to the sample type. Only the analysis of As on filter gave a higher
repeatability of up to 20 %. The reproducibility was higher than the repeatability.
Furthermore, for a majority of participants the between day variability, determined on
three different days with different calibration, was higher than the within-day variability
of measurements. These two observations suggest that it should be still possible to
improve the quality of measurements by implementing more stringent procedures of
quality control.
More detailed discussions and results can be found in EUR 23219 EN 2008
S1
S1
S1
S2
S3
S4
S5
S5
S5
As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb As Cd Ni Pb
2 < z < 3
NRL0
-2 < z < 0
NRL1
Z > 3
NRL2
ED-XRF
NRL3
ICP-MS
NRL4
GF-AAS
NRL5
ICP-OES
NRL6
VOLTAMMETRY
NRL7
No results
NRL8
No data treatment
NRL9
NRL10
NRL11
NRL12
NRL13
NRL14
NRL15
NRL16
-3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3 0 3
69
LCSQA / INERIS
S4
EC z-score
DQO S8
50% S9
S5
S6
S7
S10
S11
NPL
S3
ERLAP
S2
EMPA
S1
S12
S13
4
3
2
-3
IMI
ISCIII
GGD
UNIMIB
VMM/UGent
LRA
UBA D niosh
NERI
JRC
UBA D eusaarII
-2
NCSR-D
-1
UBA A
CHMI
-4
% deviation
100
50
-50
-100
24 h average over time
Fig 6.9 deviation from ERLAP reference PM10 value for 16 campaigns
70
References
1. International Vocabulary of metrology Basic and general concepts and
associated Terms (VIM), JCGM 200:2008 (see www.bipm.org)
2. M.J.T. Milton and T.J. Quinn, Primary methods for the measurement of amount of
substance, Metrologia, 2001, 38, 289-2961.
3. J. Viallon et al; A study of systematic biases and measurement uncertainties in
ozone mole fraction measurements with the NIST Standard Reference
Photometer; Metrologia 43 (2006) 441 450;
4. Demonstration of Equivalence of Ambient Air Monitoring Methods:
http://ec.europa.eu/environment/air/quality/legislation/assessment.htm
4. A.S. Brown, R.J. Brown, W.T. Corns, P.B. Stockwell, Establishing SI traceability for
measurements of mercury vapour, Analyst 133, (2008), 946 953;
6. Organisation of Intercomparison Exercises for Gaseous Air Pollution for EU
National Air-Quality Reference Laboratories and Laboratories of the WHO Euro
Region.
7. International Key Comparison CCQM-K26a and Pilot Study CCQM P50a (NO):
http://kcdb.bipm.org/AppendixB/appbresults/ccqm-k26.a/ccqmk26.a_final_report.pdf
8. International Key Comparison CCQM-K26b and Pilot Study CCQM P50b (SO2),
http://kcdb.bipm.org/AppendixB/appbresults/ccqm-k26.b/ccqmk26.b_final_report.pdf
http://www.bipm.org/utils/common/pdf/final_reports/QM/P28/CCQM-P28.pdf
10. EC Intercomparison of VOC measurements between National Reference
Laboratories (AQUILA Network), European Commission Joint Research Centre
Report EUR 23529 2008;
11. EURAMET Comparison of multi-component ambient VOC measurements (886)
http://kcdb.bipm.org/AppendixB/appBresults/EUROMET.QM-S3/EUROMET.QMS3_Final_Report.pdf
http://kcdb.bipm.org/AppendixB/appbresults/ccqm-k7/ccqm-k7_final_report.pdf
13. International Comparison CCQM-K10: Benzene, Toluene and Xylene (BTX) in
Nitrogen at regulatory levels 2001:
http://kcdb.bipm.org/AppendixB/appbresults/ccqm-k10/ccqm-k10_final_report.pdf
14. International Comparison CCQM-K22: Volatile organic compounds in air 2003:
http://kcdb.bipm.org/AppendixB/appbresults/ccqm-k22/ccqm-k22_final_report.pdf
71
ANNEX 1:
General
Background and objectives
72
The objectives of this initiative are to merge these activities and therefore to:
o prevent duplication of participation,
o optimize the value of the IEs to the participants,
o ensure the comparability and accuracy of results obtained beyond the current EU
borders and
o optimize the technical capabilities of the participating laboratories.
IE procedure
These IEs are carried out according to the principles of ISO Guide 43-1 1 (1997).
Organiser and participants
The IEs are organized by the European Commissions, DG JRC, European Reference
Laboratory for Air Pollution (ERLAP), in collaboration with the WHO European
Centre for Environment and Health (WHO/ECEH, Bonn Office) and WHO CC.
Reference documents
Registration form
Questionnaire
Reporting form
Complaint form (LAB-REC-0310)
The IEs are usually organised twice a year. All NRLs are required to participate at least
once every three years. The same is recommended for the other laboratories.
The duration of an IE is about 3 days. However, additional time is needed for
installation, warming up and dismantling of the equipment.
Three IE facilities are currently available:
1) ERLAP
Joint Research Centre IES, T.P. 441, I 21020 Ispra (VA)
Contacts:
Annette Borowiak (annette.borowiak@jrc.it)
Friedrich Lagler (friedrich.lagler@jrc.it)
2) LANUV NRW, Wallneyer Str 6, D 45133 Essen
Contacts:
Ulrich Pfeffer (ulrich.pfeffer@lanuv.nrw.de)
3) UBA, Paul-Ehrlich Str 29, D 63225 Langen
Contacts:
Volker Stummer (volker.stummer@uba.de)
Hans-Guido Muecke (hans-guido.muecke@uba.de)
Invitation, measurements, communication, reporting and deadlines
ISO/IEC Guide 43-1:1997, Proficiency testing by interlaboratory comparisons - Part 1: Development and
operation of proficiency testing schemes, ISO, Geneva, Switzerland.
73
Measurements
The measurement methods to be used by the NRLs are those specified as reference
methods in the AQ Directives (or in alternative the ones that have formally been
recognized as equivalent). For the other laboratories, national measurement methods
may be used, but reference methods are recommended.
The participants must bring their own complete measuring equipment that is needed for
the analysis and data acquisition of the test gases, including, where possible calibration
facilities.
Generation of test gases
It is possible that not all the compounds listed below will be tested at each IE.
At least three concentration steps will be generated per compound.
The following table indicates the concentration ranges of interest for the
intercomparison exercise (chosen as 75 % of the measurement ranges defined in the EN
standards 2 ):
Compound SO2
NO
NO2
CO
3
3
3
Conc. min 0 g/m
0 g/m
0 g/m
0 mg/m3
Conc. max 750 g/m3 900 g/m3 375 g/m3 75 mg/m3
O3
Benzene
3
0 g/m
0 g/m3
400 g/m3 50 g/m3
EN 14212 (2005), Standard method for the measurement of the concentration of sulphur dioxide by ultraviolet fluorescence
EN 14211 (2005) Standard method for the measurement of the concentration of nitrogen dioxide and nitrogen monoxide by
chemiluminescence
EN 14625 Standard method for the measurement of the concentration of ozone by ultraviolet photometry
EN 14626 Standard method for the measurement of the concentration of carbon monoxide by nondispersive infrared spectroscopy
EN 14662-3 (2005) Ambient air quality - Standard method for measurement of benzene concentrations - Part 3: Automated
pumped sampling with in situ gas chromatography;
74
Each concentration level will be normally generated for a minimum of 2 hours. Shorter
durations may apply for particular studies. During an IE, the organiser may also
introduce interferences into the test gas, i.e. to check for compliance of equipment
according to the EN standards. Other tests regarding certain performance characteristics
may also be performed during the IE. These possible tests will be detailed into the
invitation to participate to IEs.
Reporting of the measurement results
Each participating laboratory is required to deliver three 30-minute averaged values (in
nmol/mol for NO, NO2, NOx, SO2, O3 and benzene while in mol/mol for CO) and
their associated uncertainty (obligatory for NRLs and recommended for others) for each
compound and concentration.
Each participating laboratory is also required to complete the questionnaire inquiring on
traceability, implemented practice concerning calibrations, measurements and
uncertainty evaluation. Further reporting requirements may be sent with the invitation.
Evaluation scheme
General
The evaluation of the results of the IE will be carried out according to the ISO Guide
43-1 and ISO 13528. Proficiency of participating laboratories will be evaluated by two
methods:
1. The z-score method 3 will be used to demonstrate the capacity of NRLs to comply
with the uncertainty requirements for calibration gases stated in the relevant EN
standards (which are consistent with the data quality objective DQO of the
European Directives). Other criteria may apply to laboratories other than NRLs.
The z-score will be evaluated for all participants of the IE and for all runs having a
reference value. For example the interference tests (see hereafter) may not have
reference values and therefore cannot be treated using the z-score method.
2. The En-number method will be used to demonstrate that the difference between
participating laboratories results and assigned values remains within participating
laboratories claimed uncertainties and the uncertainty of assigned values. The Ennumbers are calculated for all participants reporting uncertainty of measurements,
this latter parameter being mandatory for NRLs.
Beside proficiency of participating laboratories also repeatability and reproducibility of
standardized measurement methods will be evaluated according to ISO 5725-2 4 . This
group evaluation will be used as an indicator of the trend of the quality of
measurements from one IE to other ones (ISO 13528 6.7).
In some IEs, tests of interference on the response of analysers will be performed. The
reporting of results of these tests is only informative and is not foreseen is this
document.
Assigned values for proficiency evaluation
Generally the measurements of ERLAP will be used as assigned values (X) of IEs. The
assigned values of tested concentration levels will be derived from a calibration against
the certified reference values of the CRMs (ISO 13528 . 5.4) and will be confirmed by
3
ISO 13528 (2005) Statistical methods for use in proficiency testing by interlaboratory comparisons. ISO, Geneva, Switzerland.
ISO 5725:1994, Accuracy (trueness and precision) of measurement methods and results -- Part 2: Basic method for the
determination of repeatability and reproducibility of a standard measurement method, ISO, Geneva, Switzerland
4
75
comparison to robust averages (ISO 13528 5.7). If ERLAP measurements fail to pass
this conformation test or if the IE will take place far from Ispra and ERLAP will not be
able to implement its optimum measurement capability, the assigned values will be
calculated as the robust averages (ISO 13528 5.5) from a subset of expert NRLs. As
expert NRLs will be regarded laboratories that participate to BIPM CCQM GAWG key
comparisons and/or are accredited with appropriately small uncertainty. For each IE the
list of participating expert NRLs will be given in step 5 of indicative time table ( 4).
The uncertainty of the assigned value will be calculated as combined uncertainty of the
ERLAP measurement uncertainty and the possible lack of homogeneity among the
different position on the testing bench. However, if the assigned value is calculated
according to (ISO 13528 5.5) instead of using ERLAPs value then the uncertainty
will be calculated with the equation 1 (ISO 13528 5.5.2):
uX =
1.25
p
2
i
(1)
where p is the number of expert NRLs and ui are their standard uncertainties.
The calculation of the assigned value and its uncertainty will be documented and made
available in an annex to the report of the IE.
z-score
z' =
xi X
(2)
p2 + u X2
where xi is a participants value, X is the assigned value, p is the fitness-for-purposebased standard deviation for proficiency assessment and uX is the standard
uncertainty of the assigned value.
In the NO2, SO2, CO and O3 EN Standards the uncertainties for calibration gases used
in ongoing quality control are prescribed. In fact, it is stated that maximum permitted
expanded uncertainty for calibration gases at the calibration point (75% of calibration
range) is 5% and that zero gas shall not give instrument reading higher than the
detection limit. However no criteria for detection limits are prescribed. As one of the
tasks of NRLs is to verify the accuracy of zero gas and calibration gas mixtures the
standard deviation for proficiency assessment (p) is derived in a fitness-for-purpose
manner from requirements given in the EN standards, where in place of detection limits
criteria the specifications for purity of zero gas used in type approval as defined in EN
Standards are taken. This general reference to the EN standards can not be made for
benzene, where measurement method used by NRLs for verification purposes can differ
from the method used at IE. Therefore for benzene p is set to 6% at the calibration
point.
Over the whole measurement range, p is calculated by linear interpolation between the
value at the calibration point and zero. The linear function parameters (a,b) of p are
given in table 1. Figures 1 to 6 give the absolute and relative values of p over the
concentration range for each compound.
76
p (calibration
pnmol/mol=a[Assigned value]nmol/mol+b
point)
a
b
nmol/mol
nmol/mol
7.1
0.0215
1
1613
0.0234
100
4.7
0.0197
1
18.1
0.0236
1
4.9
0.0199
1
0.7
0.0566
0.04
p (zero)
nmol/mol
1
100
1
1
1
0.04
SO2
CO
O3
NO
NO2
Benzene
10
10
0
0
50
100
150
200
250
p relative (%)
p (nmol/mol)
0
300
Figure 1
2.0
10
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
p relative (%)
p (mol/mol)
0
0
10
20
30
40
50
60
70
Figure 2
77
10
10
0
0
50
100
p relative (%)
p (nmol/mol)
0
200
150
Figure 3
20
10
18
16
14
12
10
0
0
100
200
300
400
500
600
700
p relative (%)
p (nmol/mol)
0
800
Figure 4
10
10
0
0
20
40
60
80
100
120
140
160
180
p relative (%)
p (nmol/mol)
0
200
Figure 5
78
1.0
9
8
0.8
6
5
4
0.4
p relative (%)
p (nmol/mol)
7
0.6
0.2
1
0.0
0
0
10
11
The normalized deviations, according to ISO Guide 43-1 (ISO, 1997), will be used to
evaluate whether the differences between the results of laboratories that reported
expanded measurement uncertainty together with their measurement results and the IE
reference value remained within the stated uncertainties. The normalised deviations are
calculated using the following equation:
En =
xi X
(3)
U x2 + U X2
The procedure laid down in ISO 5725-2 will be implemented in order to evaluate the
repeatability and reproducibility of the measurement methods. Data consistency and
outlier tests will be performed and relationship between r and R and the concentration
levels of the IE test will be investigated.
Assessment
The z-score evaluation allows the following criteria to be used for the assessment of
the results:
o -2 z 2 are designated satisfactory. Approximately 95 % of z-scores should fall
between 2 and +2.
o -3 z < -2 or 2 < z 3 are designated questionable. They are expected about 1
time out of 20.
o z < -3 or z > 3 are designated unsatisfactory. Scores falling in this range are very
unusual and are taken to indicate that the cause of the event should be investigated
and remedied.
79
The IEs are organized in order to provide the NRLs with the possibility of comparing
their results and test their proficiency. As the quality of the NRLs measurement is
connected to the data quality of the Member State, the European Commission requires
satisfactory results within the data quality objectives to be obtained. In the case of
NRLs overall unsatisfactory results of z-score evaluation (one unsatisfactory or two
questionable results per parameter (ISO 13528 7.4.2)) the EC requires to repeat
participation to the next IE in order to demonstrate remediation measures. In case of
failing participation to the IEs for more than 3 consecutive years, the JRC will inform
DG-Env.
Complaints
Should be sent in writing to the organiser of the IE within 5 months of its completion
(annette.borowiak@jrc.it or friedrich.lagler@jrc.it).
Costs
The costs of participation by the NRLs in the IEs will be covered by the NRLs
themselves. Other laboratories may apply to the WHO CC or the JRC for financial
assistance.
80
ANNEX 2:
The original requirements for implementing quality assurance procedures for ambient
air quality monitoring at an EU Member State level were specified in Council directive
96/62/EC on ambient air quality assessment and management (the so-called
Framework directive). Article 3 of this directive requires that:
The Member States shall designate at appropriate levels the competent authorities
and bodies responsible for, for example, ensuring the accuracy of measurement by
measuring devices and checking the maintenance of the accuracy of such devices, by
internal quality controls carried out in accordance, inter alia, with the requirements of
European quality assurance standards.
The Framework directives requirements as to which European quality assurance
standard(s) should be used, however, were not made explicit. At the time that this
directive was published in 1996, there were two types of such quality assurance (QA)
standards in existence:
1. Standards for the implementation of overall quality systems in organisations developed by the International Standardization Organisation (ISO), particularly
those in the ISO 9000 series of standards initially ISO 9001:1994 & ISO
9002:1994, which were then both superseded by ISO 9001:2000. These standards
apply to the general and overarching quality system required within a particular
organisation so as to ensure the ongoing quality, acceptability, and consistency, of
the products and/or services that the organisation was involved with. These
activities may cover, for example, a particular manufacturing process, or a specific
type of non-technical service to customers. These standards are not intended to
cover in any level of detail the scientific technical and experimental activities of that
organisation, or to provide guidance on, for example, how measurements may be
carried out, or how the uncertainty of such measurements should be determined by
the organisation.
2. Standards developed initially at a national level in many countries to ensure that a
specific technical and experimental activity, particularly those that are
measurement related, are carried out in a consistent quality-controlled manner so
as to ensure that the end result is delivered with specified quality, and thus they
result in a defined and required accuracy (and hence the measurement
uncertainty) of all the results produced. These standards were produced by the
European standardization body, the European Committee for Standardization
(Committee European de Normalisation, known as CEN), and published originally
as the EN 45000 series of standards.
There was, however, no overall consensus for some time on which of the two types of
quality assurance standards the Framework directive intended to refer to. It could be
argued that the choice of the standard was implied, since the Framework directive
refers to measurements and to technical activities and to European standards.
From this it could be judged that the EN 45000 series was the most appropriate.
81
Additionally, there was some discussion as to how the quality assurance standards
should be implemented for the NRLs and other related laboratories, since this was
also not made explicit within the Framework directive. For example:
-
Or should the quality assurance standard and its procedures and systems as it is
established by the laboratory be determined by assessment using an independent
body such as a national accreditation body or similar?
These issues were not fully resolved. The subsequent activities discussed in Chapter 3
made this unnecessary.
82