You are on page 1of 72

NUCL-5005: Special Topics – Principles of Nuclear Facility Decommissioning

Site Characterization:
Data Quality
Objectives
PMod4
Presenter: Dr. G. Harvel

Note: Additional details provided in Syllabus in CANVAS

Energy Systems
& Nuclear Science
Safety Protocol: At OTU and where ever
you are…
Today’s safety moment
• Fire
• Medical • Student Choice
• Security

Note: Examinable material

2
Learning Outcomes
In this module, the participant will have studied
the following:
• Use MARSSIM type surveys for facility
characterization
• Understand waste Characterization by CSA
N292
• Identify recommended sampling and analytical
methods
• Develop a characterization survey plan and
report

Note: Information describe in this module is primarily from EPA QA/G-4, CSA
N292.8(In Preparation), MARSSIM, ASME Decommissioning Handbook

3
DQO Process
• DQO stands for Data Quality Objectives
• Process developed by the EPA as documented in
– EPA QA/G-4

• Adopted for use by many organizations/countries to


assist in Waste Characterization

• Canada adopting its use as per CSA N292.8


– (under development)
• Use of the DQO Process leads to efficient and effective
expenditure of resources; consensus on the type,
quality, and quantity of data needed to meet the project
goal; and the full documentation of actions taken during
the development of the project

4
DQO Process
• The DQO Process is a series of logical steps that guides
managers or staff to a plan for the resource-effective
acquisition of environmental data.
• It is both flexible and iterative, and applies to both decision-
making (e.g., compliance/non-compliance with a standard) and
estimation (e.g., ascertaining the mean concentration level of
a contaminant).
• The DQO Process is used to establish performance and
acceptance criteria, which serve as the basis for designing a
plan for collecting data of sufficient quality and quantity to
support the goals of the study.
• Use of the DQO Process leads to efficient and effective
expenditure of resources; consensus on the type, quality, and
quantity of data needed to meet the project goal; and the full
documentation of actions taken during the development of the
project.

5
The DQO Process is a flexible planning tool that can be
applied in a graded manner depending on the complexity of
the situation. This graded approach defines data quality
requirements, the risk of making a decision error based on the
data collected, and the consequences of making such an error.

Key to the DQO Process is developing statistical hypothesis tests for


implementing decision rules using measurement data. There are two
problems to address in decision making:
a) infinite resources are not available to apply to the question being asked;
and
b) it is usually not possible to be 100% sure that the right conclusion has
been reached (i.e., there is usually some “uncertainty” associated with the
conclusion).

6
7
STEP 1: State the Problem
Define the problem that necessitates the study; identify the planning team, examine budget,
schedule
Determine if Diesel
Generator ground is
contaminated

STEP 2: Identify the Goal of the Study Use soil samples to


State how environmental data will be used in meeting objectives and solving the problem, identify determine
study questions, define alternative outcomes contamination level.

STEP 3: Identify Information Inputs


Identify data & information needed to answer study questions

STEP 4: Define the Boundaries of the Problem How many samples?


Specify the target population & characteristics of interest, define spatial & temporal limits, scale of How deep?
inference

8
STEP 5: Develop the Analytical Approach
Define the parameter of interest, specify the type of inference, and develop the logic for drawing
conclusions from findings

Decision Making Estimation and Other Analytical


(hypothesis testing) Approaches

STEP 6: Specify Performance or Acceptance Criteria


Specify probability limits for false rejection and Develop performance criteria for new data being
false acceptance decision errors collected or acceptable criteria for existing data
being considered for use

STEP 7: Develop the Plan for Obtaining the Data Prepare


Select the resource-effective sampling and analysis plan that meets the performance criteria characterization survey
plan

9
S1: Stating the Problem
• Give a concise description of the problem
• Identify leader and members of the planning
team
• Develop a conceptual model of the
environmental hazard(s) to be investigated
• Determine resources – budget, personnel, and
schedule.

10
S1: Activities
• Describe the problem, develop a conceptual
model of the environmental hazard(s) to be
investigated, and identify the general type of
data needed:
• Establish the planning team and identify the
team’s decision makers;
• Discuss alternative approaches to investigation
and solving the problem;
• Identify available resources, constraints, and
deadlines associated with planning, data
collection, and data assessment

11
S1: Characterizing the Problem
• Conceptual Model
– Known or expected locations of contaminants;
• From historical data or preliminary surveys
– Potential sources of contaminants;
• Both past and active
– Media that are contaminated or may become
contaminated, and
– Exposure scenarios (location of human health
or ecological receptors)

• Important to include theories, assumptions,


information from previous studies that
helped solved the problem, performance of
sampling and analytical methods.

12
S1: Establishing the Team
• Project Manager, technical staff, data users,
and stakeholders
– Project manager usually also a decision maker

• Technical staff should include individuals


knowledgeable about
– Geographical layout, sampling constraints,
analyss, statistis, and data interpretation
• Data users can include analysts,
decommissioning staff
• Stakeholders are indivudals or organizations
that are directly affected by the decision or
study result
– May include public, regulators, or other groups

13
S1: Resources, Constraints Deadlines
• Identify and examine limitations that
would be present on resources and
time constraints associated with the
process of collecting data
• Practical Constraints:
– Right of entry Could affect sampling
– Seasonality collection, quality, or
accuracy
– Physical location
Training, retirements,
– Available personnel knowledge retention
– Contracts
– Deadlines

14
S1: Intended Use of Data
• Decision Making vs. Estimation
– Confirmation of compliance in
regulatory limit

– Support the development of a process

– Generate a set of estimates for later


use in planning Is this an environmental
Will data be used to survey attempting to
choose between characterize levels of
alternative actions? exposure for specific
populations or areas?

15
S1: Example
• Problem Description
– Historical data shows transformer spills in switchyard area but specific
locations and extent of condition is not known
• Team
– Electrical Project Manager (DM), Chemical Analyst (oils and PCBs), Field
Engineer, member from city health department (groundwater knowledge)
• Conceptual Model
– Most likely contaminated area is within 2 meters of the main transformer
stations and along the main drive route where components were shipped
in and out of the yard. Previous studies show the soil is sand/gravel and
thus potential for deep penetration. Rainfall is potentially moving PCBs
into the water table
• Resources and Constraints
– Deadline: within one year
– Sampling method available and approved
– Analysis laboratory will be shutdown for upgrades in 8 months
– Regulatory approval required to start sampling
– Budget available for team of 3 to conduct samples and analysis of < 75
samples

16
S2: Identify the Goal of the Study
• Identify principal study questions
• Consider alternative outcomes or
actions that can occur
• For decision problems, develop
decision statement(s), organize
multiple decisions
• For estimation problems, state what
needs to be estimated and key
assumptions

17
S2: Activities
• identify the principal study question and define
alternative actions that may be taken based upon the
range of possible outcomes that result from
answering the principal study question;

• use the principal study question and alternative


actions to make either a decision statement or
estimation statement (whichever is relevant to the
particular problem); and
• organize multiple decisions into an order of sequence
or priority, and organize multiple estimation
problems according to their influence on each other
and their contribution to the overall study goals.

18
S2: Identify the Principal Study
Question
• Helps focus the search for
information to address the problem
• Examples: Decision problems
• Does the concentration of contaminants exceed
acceptable levels?
• Does a contaminant pose a human health or ecological
risk?
• Is the contaminant concentration significantly above
background levels?

• Examples: Estimation problems


• What is the average rate of effluent?
• What is the distribution of contaminant over space and
time?
• What are the sizes of endangered species populations
within the habitat of concern?
19
S2: Alternative Actions
• Identify a series of possible actions that may
be taken once the question is answered.
– Note: no action is an alternative
• Team should confirm that the alternative
actions can solve the problem and also if
regulations would be satisfied.
• All concrete to be shipped
to waste depository
Is the Chlorine activity in
• Seal the concrete to
reactor concrete above
ensure Chlorine remains
the clearance level?
trapped inside concrete
and follow in-situ
decommissioning
• Chip away first 10 cm
layer of concrete for
disposal
• Take no action

20
S2: Developing Decision Statement
• Use principle question and alternate
actions to develop a set of decision
statements
• Determine if
(condition/issue/criteria) requires
(taking the following action)

21
S2: Developing an Estimation Statement

• Consider a range of possible outcomes associated


with estimating some unknown entity

What is the average rate of PCB effluent


in discharge channel over time?

The principal quantity to be estimated is the PCB


concentration in water. We anticipate that the PCB
concentration will normally be low and will not exceed 0.3
ppb. Regulatory limit is (0.5 ppb). Under high rainfall or
flood scenarios, samples may result in very low
measurement levels due to mixing with uncontaminated
water. Under drought conditions, the contamination level
may exceed the regulatory limit for a limited number of
days.

22
S2: Example
• Specify the primary question
– Does the concentration levels of PCBs in
the soil exceed the regulatory limit
• Determine alternative actions
– Remove soil to waste disposal site
– Decontaminate the soil to below clearance
levels
– Take no action
• Specify the decision statement
– Determine if the PCB contamination within
the soil requires disposal to waste
disposal site, soil simply cleaned to below
clearance levels

23
S3: Identify Information Inputs
• Identify and confirm:
– the types and potential sources of
information needed
– Information basis for specifying
performance or acceptance criteria;
and
– The availability of appropriate sampling
and analysis methods.

24
S3: Type of information needed

• Prepare list of characteristics that will


need to measured to address the
problem statement.
• Is information on the properties of the
media required?
– Physical, Chemical, Radiological
• Can existing data be used to make the
decision or produce the estimate?
• Do we need to collect new
measurements on environmental
characteristics?

25
S3: Issues using existing data
• Often historical data or existing surveys are available
to provide information
• The quality and accuracy of the existing data may
challenge the outcome
– Confirm adequate quality assurance and information
control
– Confirm applicability of methods used and consistency
with methods chosen for the remainder of the data set

26
S3: Basis for Performance and
Acceptance Criteria
• Identify the basis for performance and acceptance criteria to guide steps 5
and 6
• For decision problems, the analytical approach requires decision rules
incorporating some type of Action Level, a threshold to determine choice of
alternative actions as per Step 2.
• The specific information source used to determine this level is done now.
Actual numerical value needed by Step 5.

• For action level relative to background, need to determine the basis for
characterizing background.

27
S3: Considerations for identifying and
evaluating appropriate sampling and
analysis methods
• Use list of environmental characteristics that are necessary
for addressing a particular decision or estimate
• Develop list of physical sampling and analytical methods
appropriate to obtain the necessary information (If no
methods, return to step 2 and change goal)
• Decision problems: bias kept to minimum. Mitigate:
– Non-representative sampling
– Instability or contamination of sample
– Interferences and matrix effects in analysis
– Calibration

28
S3: Example
• Type of information needed
– Historical data inaccurate. Need new information to be
obtained. Soil samples and water samples from nearby
runoff are to be taken to measure PCB levels.
• Source of information
– Data from existing pilot study will be used to identify
locations of soil samples.
• How action level will be determined
– Local regulations on PCB concentrations in water will
be used to determine the action level.
• Identify appropriate sampling and analysis methods
– A leachant method will be used for the soil samples to
extract the PCBs. PCB Aroclor analysis via gas
chromotagraphy will be used to represent PCB
concentration

29
S4: Define Boundaries of Study
• Define target population of interest and
relevant spatial boundaries
• Define what constitutes a sampling unit
• Specify temporal boundaries and other
constraints on sample/data collection
• Specific smallest unit on which decisions or
estimates are made

30
S4: Define Target Population

• Total collection of sample units


Example:
Intent is to use swipes for sampling tritium
contamination in concrete surfaces where
each swipe is one sample.

Intended area is 500m2 with 1 sample


every 10m2

Total population is 50 samples

31
S4: Types of Spatial and Temporal
Boundaries
• Spatial boundaries define physical area and
general locations of samples
• Temporal boundaries describe time frame
and when samples to be taken

SPRING ONLY (Flood Season)

LATE SUMMER

32
S4: How to Determine Spatial
Boundaries
• Step 1 Conceptual Model provides input
• Define geographic area applicable for decision
making or estimation
– Entire area using latitude, longitude, altitude
(clear even if site is altered)
– Length, area, volume, legal boundaries
– Media, soil depth, water depth,
• Divide population into subset of relatively
homogeneous characteristics
– All same type of media (gravel, sand, concrete,
etc.)
– Subset has expected similar contamination level

33
S4: How to Determine Temporal
Boundaries
• Step 1 Conceptual Model provides input
• Consider temporal phenomena at site
– Weather variations,
– seasonal effects,
– equipment operation under different conditions
(temperature, moisture).
• When is decision or estimate relevant?
– May not be possible to collect data over full time
period the decision or estimate will apply
• Mainly future use applications (i.e. residual dose
to resident…)
34
S4: Constraints on Data Collection

• Access to Property
• Availability and operation of equipment
• Environmental Conditions when sampling is not
possible (high humidity, freezing temperatures)
• Availability of personnel
– Sampling team, QA/QC team, inspectors, regulators

35
S4: Define Scale of Inference

• How is the smallest unit of area, volume, or time delineated over


which data will be:
– Collected, analyzed, aggregated and interpreted

• This step can affect the estimate and uncertainty determined in


Step 6. Careful selection of size, location, and shape is required.

• Too many samples increases the cost beyond requirement

• Too few samples increases risk or potential for additional rework

36
S4: Additional Guidance on Scale

• Risk – Increase sampling in areas of future activity


where risk may be higher if incorrect decision is
made
• Technological Considerations – most efficient area of
volume that can be remediated using a selected
technology.
• Temporal Considerations – sampling at a frequency
that captures potential changes
• Financial Scale – If full scale remediation is too costly,
may need to redefine sampling for greater accuracy
on affected area
• Historical information – may identify hotspots or
sensitive areas

37
S4: Example
• Target Population
– Target population consists of all possible samples of PCB
concentrations in soil and in water.
• Spatial/Temporal boundaries and Constraints
– The spatial boundary consists of the following subsets:
general switchyard, transformer location, vehicle travel
route, and nearby drainage ditch for water samples.
– Temporal boundaries will include high water run off and
low water run off times.
– No measurements in winter as ground is frozen which may
bias results.
– Environmental Inspector required for at least one
sampling period.
• Scale of Inference
– Higher sample density in transformer subset.

38
S5: Develop the Analytical Approach

• Specify appropriate population


parameters for making decisions or
estimates
• Decision problems: choose a
workable action level
– “If..Then.. Else…”
• Estimation problems: Specify the
estimator and the estimation
procedure
39
S5: Population Parameters

• Parameter (true mean, median,


percentile) that summarizes critical
characteristic or feature of the
population
• May be based on regulatory
limit/requirements
• May be project specific considerations

• Most common is population mean


(useful for random exposure model)

40
S5: Decision – Action Levels

• Pre-Determined Action Levels


– Drinking Water Standards
– Clearance Levels

• Investigation-Specific Action Levels


– Background or specific risk standards

– Required degree of conservatism


– Threshold of concern versus more
conservative approach

41
S5: Action level – detection limits

• Document detection limit for each


potential measurement method from
step 3
• If detection limits is very close or
exceeds action level,
– Measurement method should be
changed to a more sensitive method
– Different analytical approach should be
used

42
S5: Develop Theoretical Decision Rule
• “If…Then…Else…”

• If the true mean tritium concentration on surface


exceeds clearance level, then decontaminate
surface, else release concrete to municipal waste
stream.

• If the true mean tritium concentration on surface


exceeds 3X clearance level, then break into volumes
that fit in waste container and ship to LLW site,
elseif true mean tritium concentration on surface is
between clearance and 3X clearance level, then
decontaminate surface, else release concrete to
municipal waste stream.
43
S5: Develop Specification of the
Estimator
• Specify the estimator by combining the
selected population parameter with the scale
of estimate and other population boundaries.

Example:
The study will estimate the total annual concentrations
of mercury in the effluent from the facility within 500
meters of the discharge.

44
S5: Example
• Specify Appropriate Action Level
– The clearance level for Cs-137 is 0.1
Bq/g. Action level is set to clearance
level.
• Specify Theoretical Decision Rule
– If the mean concentration of Cs-137 in
the soil exceeds 0.1 Bq/g, then the soil
shall be remediated, else the soil is
clear for release.

45
S6: Specify Performance or
Acceptance Criteria
• For decision problems, specify the
decision rule as a statistical hypothesis
test, examine consequences of making
incorrect decisions and place acceptable
limits on the likelihood of making
decision errors.

• For estimation problems, specify


acceptable limits on estimation
uncertainty.
46
S6a: Statistical Hypothesis Testing
• Specific Probability Limits for false rejection
and false acceptance
• What are the different sources of error in the
data?
• Statistical Sampling Error
– Influenced by inherent variability on the
population over space and time, sample collection
design, and number of samples taken. Sample
collection does not fully represent population.
Measurement Error (Physical Sampling)
– Influenced by imperfections in the measurement
and analysis.

47
Total Study Error
(Total Variability)
ERROR
SOURCES Measurement Error
Sampling Error
(Measurement
(Field Variability)
Variability)

Inherent Variability Sampling Design

Stratification Sampling Frame Selection

Sampling design and Homogenization Sampling Unit Definition


accurate Selection Probabilities
measurements are key
elements in reducing Number of Samples

total error study


Total Study Error Total Study Error Total Study Error
(Total Variability) (Total Variability) (Total Variability)

Keep track of any


Support
assumptions Volume/Mass
Preservation Preparation

Sample
Packaging Subsampling
Delineation

Sample
Transport Extraction
Extraction

Analytical
Storage
Determination

Labeling Data Reduction


48
S6a: Possibility of incorrect decisions

• Must accept there is some likelihood outcome may


lead to incorrect conclusion – decision error.
• Establish baseline condition – met clearance level.
Alternate condition – exceeded clearance level

• Insufficient evidence from data to indicate baseline is


false, conclude that baseline condition is true
• Sufficient evidence from data that baseline is false
beyond a reasonable doubt, conclude alternate
condition is selected
• Standard presumption is in favour of the baseline

49
S6a: Possibility of incorrect decisions

Purpose of Step 6a process is to arrive at upper limits on the


probabilities of each of the two decision errors that are acceptable

False Rejection – Type I error. Probability error term (α)

False Acceptance – Type II error. Probability error term (β)

50
Action
Level

False (β)
Acceptance

False
Rejection

(α)

51
S6a: Defining Baseline Condition

• Set to Regulatory limit or some


conservative lower bound
– Unconditional Clearance Level
– Can add Gray Region
• For no regulatory limit, set the
action level based upon expected
consequences
– No limit but research study suggests a
trend to a harmful level

52
Gray
Region
Action
Level

Gray Region Considered equivalent to Action Level

53
S6a: Establishing Tolerable Limits on
Decision Error Probabilities
• False Rejection decision error limit (α)
• False Acceptance decision error limit (β)
– Most stringent error is usually 1% or 0.01
– Use as starting point and consider consequences of
error
– If consequences are low, possible to increase the
allowable error limit
Example: Action Level = 100; (α) = (β) = 0.01. Gray Region narrow.
Consequence of False Rejection is increased disposal to waste site.
Conservative.
Consequence of False Acceptance is increased exposure to
environment. Consequence at parameter value 100-120 is low.
Consequence at parameter level >120 moderate. Gray region could
be extended up to 120 and allow for a higher probability of error.

54
S6b: Estimation

• Estimation can use many different statistical


parameters to define the population

• Mean
• Median
• Upper percentile, upper confidence limit
• Slopes
• Ratios
• Rates of Processes
• Estimates of Toxicity
• Spatial Contours
• Population size, growth rate, decay rate

55
Median Mean

Action Level

-1 Sigma +1 Sigma

56
S6b: Uncertainty

• Measured Value +/- standard error


– Mean value 0.5 Bq/g
– Standard Error 5%

• Interval instead of point value


• Confidence interval/limit; tolerance
interval/limit; prediction interval/limit
– Mean value 0.5 Bq/g
– Confidence interval is +/- 0.05 for a 95%
confidence level

57
S6b: Standard Error

• Function of:
– Amount of data available
– Underlying distribution (normal or not)
– Variability in the data

• Value is able to set goals on the standard


error that will help limit amount of data
collected
– Maximum allowed error +/- 15%

58
S6b: Intervals

• Confidence Intervals
– Descries population mean and standard
deviation
• Tolerance Intervals
– Contain the population that has a
significant probability
– Useful for non-normal distributions
• Prediction Intervals
– Generated from the data to predict a
future mean and standard deviation

59
S6: Example
• Setting baseline and alternative conditions
– Will use a statistical hypothesis test. PCB in soil guidelines give 0.5
mg/kg and 1.3 mg/kg for agricultural and residential/parkland use
respectively.
– Baseline condition is soil is not hazardous as PCB concentrations will be
less than 0.5 mg/kg. Alternative condition is that the site exceeds
allowable concentrations and is hazardous.
• Determine impact of decision errors
– False acceptance – actually hazardous. Consequence is that users of end
state will have higher health risk due to higher levels of PCB
concentrations in the soil.
– False Rejection – actually not hazardous. Additional remediation costs
will incur.
• Specify the ‘gray region’ for the Decision Performance Curve
– Gray region extended to 0.7 mg/kg. Guidelines do not show significant
risk at these values and end use is not necessarily targeted for
agricultural application.
• Set the tolerable decision error limits
– False acceptance decision error limit of 0.05 (5%)
– False rejection decision error limit of 0.15 (15%)

60
S7: Develop Detailed Plan for Obtaining
Data
• Compile all information and outputs
from Steps 1 through 6
• Use information to identify
alternative sampling and analysis
designs appropriate for use
• Select and document a sampling
design that will yield data to achieve
performance or acceptance criteria
More detail can be found in EPA QA/G-5S and EPA QA/G-9S

61
S7: Types of Information Required
• Your objectives and intended use of the data
• Outputs from Steps 1 through 6 of the DQO
Process
• Background information on the problem
– Site properties, technical characteristics of
contaminants/media, regulatory requirements,
known spatial/temporal patterns
• Expected variability for the data based upon
similar studies
• Preliminary information on the underlying
distribution of the data that may impact
calculations

62
S7: Two Types of Sampling Design

• Probability Based Sampling Design


– Each sampling unit has a known probability of being selected
• Judgmental Sampling Design
– Sampling units not assigned a known probability but selected
at discretion of the team

• Statistical inference techniques require a probability-


based sampling design
– Samples are random to have random uncertainty

• Judgmental sampling has value when known historical data


is available

63
S7: Factors affect Sampling Designs

• Goal is balance amount of data with measurement


performance for cost-effective design.
– Larger number of samples using cheaper
measurement methods may help with large variability
• Sampling design should specify
– Sample selection technique
– Sample type
– Number of samples
– Number of Analyses per sample
• Alternate designs can be developed by varying the
number of spatial/temporal locations of samples,
number of replicate analyses, sample type or
technique.

64
S7: Factors affect Sampling Designs

• Reducing Variability in Population reduces


– (a) measured uncertainty or range
– (b) amount of data needed to obtain error targets
• Can divide population into strata (step 4)
– Area proximal to incinerator, diesel generator
– Area with higher exposure to weather, flood paths
• Stratification allows:
– Reducing complexity of the problem
– Reducing variability with strata
– Improving efficiency of sampling
• May be difficult to establish basis for strata and may
lead to over-stratification – large increase in total
samples

65
S7: Representation
• The extent to which measurements actually
reflect sampling unit.
• Degree to which samples represent the
population.

• Use classical probabilistic sampling designs


Different regions of total sample area have been used differently.
Split into Strata for each region of use provides better
representation of the data

66
Total Population

Mean: 54
σ = 26 Action Level
100

Strata 1 Population

Mean: 49
σ = 23

67
S7: Determine Amount of Data

• Sample size relies on estimate of variability


in the data
• Pilot study
• Results from a similar study
• Even accuracy cannot be obtained, then
ensure conservative sample size

EPA has Decision Error Feasibility Trials (DEFT) software to assist


with this step
Alternative is Visual Sample Plan Tool (VSP)

68
S7: Revisiting the Plan
• If the sampling plan does not produce
the desired result, it can be adjusted
by considering the following:
– Increase the tolerable limits on decision
errors
– Increase the funding for sampling and
analysis
– Change the boundaries of the study
– Relax project constraints

69
S7: Requirements documenting
the sampling plan
• Design elements include:
– Number of samples,
– Sample type (composite vs. individual),
– General collection techniques,
– Physical sample (Amount of material to be collected),
– Sample support (Area, quantity that each individual sample
represents),
– Sample locations (surface coordinates and depth) and how
selected,
– Timing considerations of collection, handling, and analysis,
– Analytical methods, and
– Statitical sampling scheme

70
Iterations may be required
DQO

DQA
From EPA-QA/G4

71
What did we learn?

In this module, the participant is able to:


• Describe the DQO process developed by
the EPA
• Explain how the DQO process could be
used to support characterization surveys
and planning

72

You might also like