The Environment Agencys

Monitoring Certification Scheme

(MCERTS)
- and recent CEN developments
Richard Gould
Monitoring & Assessment Process
Environment Agency

Outline
z
z
z
z
z
z

Introduction words and confusion

Background
Legal drivers
Monitoring and performance standards
Alignment and future European scheme
Existing model for verification and approval

Introduction
z
z
z
z
z
z

Environment Agency for England and Wales

Europes biggest environmental regulator
Industrial air emissions, waste, water
Flood defence, contaminated land, ecology
Monitoring is essential for control
Monitoring and assessment

z
z

Guidance and training on emissions monitoring

Application of standards
Compliance assessment and auditing

CEN and ISO standards for monitoring and quality

assurance
Monitoring Certification Scheme

Type testing and approval scheme for Automated Measuring

Systems (AMS)

Big words and confusion?

z
z
z
z
z
z
z

Verification
Certification
Approval
Type-testing
Validation
Evaluation
All the words have the same practical
meaning?

Proof of performance

It does what it says on the tin

Google 502,000 examples

Proof of performance Stage 1

Testing

Super battery charger

Fully charged battery = very quick

Certification Stage 2
Assess
Manufacturing
processes

Official
Approval

Background - Quality of Monitoring

Industrial emissions

Monitoring systems

A hazardous waste treatment site

Legal demands
z
z

Laws to control pollution

Industrial operators have permits

z
z
z

Processes have emission limit values

Processes must be controlled
Operators must monitor emissions

z
z

Permits = the law applied to the process

Independent checks too

Compliance assessment

So how do we know that the monitoring is any good?

Are the results real and valid?

Monitoring systems

Equipment

Monitoring systems

Monitoring systems

Provisions for monitoring

Reference tests

Legal drivers
z
z
z

EU Directives - incineration & combustion

Automated measuring systems (AMS)
Uncertainty budgets (95% confidence
intervals)

Accuracy and precision

Use CEN, ISO, national/other international

standards and methods

Performance requirements within monitoring

standards

Accuracy and precision

Inaccurate
Imprecise

Accurate
Imprecise
(random errors)

Inaccurate
Precise
(bias)

Accurate
Precise

Margins of error

95% CI

ELV

Past problems
z

AMS varied in quality

z
z
z
z

Very good to very bad

Variable accuracy and precision

Poor reliability
Unreliable data
AMS did not always do what they said on the
tin

Solution
z

Apply performance requirements

Different grades of performance requirement

Testing (Verification)
Certification (Approval)

Requirements for all test laboratories

Higher standards for more demanding applications

Divisions of requirements

AMS with proven performance

Reinforce with laws

Standards, accreditation

Standards applied through MCERTS

Performance specifications
z
z
z
z
z
z
z
z
z
z
z

Linearity
Repeatability
Cross-sensitivity
Response time
Detection limit
Environmental conditions (controlled chamber)
Vibration (test rig)
Reproducibility
Availability and maintenance interval
Zero and span drift
Comparison with a reference method in a field test

Zero and span

50,0

y / mg/m 3

40,0

Drift ?

30,0

20,0

10,0

0,0
0,0

10,0

20,0

30,0

x / mg/m

40,0

50,0

Standards - testing and

certification
Issue

Standard

Quality of testing

ISO 17025

AMS performance testing ISO and

CEN
standards
Product certification
EN 45011
Measurement uncertainty ISO 14956

Evaluation the MCERTS model

Laboratory tests

3 month field tests

Certification

Manufacturer audit
Surveillance

Paired testing laboratory tests

z
z
z

Artificial stack
Gas mixing systems
Environmental controls

Field testing comparison with

a reference method
Check of emissions, and
AMS-readings, using a
standard reference
method (SRM)
SRM yields a true
reading (or as close as
we can get)

SRM -vs- AMS data

- a good agreement

CEM and SRM

data

SRM -vs- AMS data - a good

agreement
16
14

y = 0.9572x - 0.0598
R2 = 0.9989

12
10

SRM
8

Plot of half-hourly
averages for SRM
versus AMS

6
4
2
0
0

10

-2

AMS

12

14

16

18

SRM -vs- AMS data - a poor

agreement

AMS
and SRM
data

Time

Surveillance - Design changes

z
z
z

Stage 1 - type testing; verification

Stage 2 - Certification; approval
Stage 3 - Continuing surveillance
control of design changes
control of manufacturing reproducibility
maintenance of performance in modified designs

MCERTS expansion

MCERTS expansion
z

Model applied to:

Continuous ambient monitoring systems (CAMs)
Continuous water monitoring systems (CWMs)

MCERTS testing schemes:

Manual stack emissions monitoring
Flow monitoring verification
Chemical testing of soils

European developments

European context
z

Germany type-approval scheme for over 25 years

z
z
z

AMS for stack emissions and ambient air

World leaders in testing and verification for AMS
UK and Germany aligned their schemes in 2002
Mutual Recognition

Italy
France
CEN member states

Updating standard
Unified performance specifications
Standards for testing AMS
International framework for testing and certification
CEN TC 264

CEN TC 264
Standard

Scope

prEN 15267-1

Framework for testing

(verification) and certification
(approval)

prEN 15267-2

Quality management system for

AMS design and manufacturing
control

prEN 15267-3

Stack-AMS performance
specifications and test
procedures

prEN 15267-4

Ambient-AMS performance
specifications and test
procedures

Salient points
z
z

MCERTS established in 1998

Based on international standards for:

Triggers:

Performance specifications, testing, design control and

quality assurance
Laws, need to improve quality of monitoring

International developments:

Aligned with the German scheme in 2002

European co-operation
Basis of new CEN standards (2007-8)

Model for verification and approval

www.mcerts.net

Lessons we have learned

z

Use standards if available

Make standards performance based

z
z

black-box approach
Input-output model
Do not stifle innovation

Provide and apply performance standards for test

laboratories
Strengthen through legislation

Benchmarks for performance

Framework for flexibility

Voluntary systems are weaker

Practicality, not academic exercises

Keep costs down for manufacturers