Professional Documents
Culture Documents
Site Characterization:
Data Quality
Objectives
PMod4
Presenter: Dr. G. Harvel
Energy Systems
& Nuclear Science
Safety Protocol: At OTU and where ever
you are…
Today’s safety moment
• Fire
• Medical • Student Choice
• Security
2
Learning Outcomes
In this module, the participant will have studied
the following:
• Use MARSSIM type surveys for facility
characterization
• Understand waste Characterization by CSA
N292
• Identify recommended sampling and analytical
methods
• Develop a characterization survey plan and
report
Note: Information describe in this module is primarily from EPA QA/G-4, CSA
N292.8(In Preparation), MARSSIM, ASME Decommissioning Handbook
3
DQO Process
• DQO stands for Data Quality Objectives
• Process developed by the EPA as documented in
– EPA QA/G-4
4
DQO Process
• The DQO Process is a series of logical steps that guides
managers or staff to a plan for the resource-effective
acquisition of environmental data.
• It is both flexible and iterative, and applies to both decision-
making (e.g., compliance/non-compliance with a standard) and
estimation (e.g., ascertaining the mean concentration level of
a contaminant).
• The DQO Process is used to establish performance and
acceptance criteria, which serve as the basis for designing a
plan for collecting data of sufficient quality and quantity to
support the goals of the study.
• Use of the DQO Process leads to efficient and effective
expenditure of resources; consensus on the type, quality, and
quantity of data needed to meet the project goal; and the full
documentation of actions taken during the development of the
project.
5
The DQO Process is a flexible planning tool that can be
applied in a graded manner depending on the complexity of
the situation. This graded approach defines data quality
requirements, the risk of making a decision error based on the
data collected, and the consequences of making such an error.
6
7
STEP 1: State the Problem
Define the problem that necessitates the study; identify the planning team, examine budget,
schedule
Determine if Diesel
Generator ground is
contaminated
8
STEP 5: Develop the Analytical Approach
Define the parameter of interest, specify the type of inference, and develop the logic for drawing
conclusions from findings
9
S1: Stating the Problem
• Give a concise description of the problem
• Identify leader and members of the planning
team
• Develop a conceptual model of the
environmental hazard(s) to be investigated
• Determine resources – budget, personnel, and
schedule.
10
S1: Activities
• Describe the problem, develop a conceptual
model of the environmental hazard(s) to be
investigated, and identify the general type of
data needed:
• Establish the planning team and identify the
team’s decision makers;
• Discuss alternative approaches to investigation
and solving the problem;
• Identify available resources, constraints, and
deadlines associated with planning, data
collection, and data assessment
11
S1: Characterizing the Problem
• Conceptual Model
– Known or expected locations of contaminants;
• From historical data or preliminary surveys
– Potential sources of contaminants;
• Both past and active
– Media that are contaminated or may become
contaminated, and
– Exposure scenarios (location of human health
or ecological receptors)
12
S1: Establishing the Team
• Project Manager, technical staff, data users,
and stakeholders
– Project manager usually also a decision maker
13
S1: Resources, Constraints Deadlines
• Identify and examine limitations that
would be present on resources and
time constraints associated with the
process of collecting data
• Practical Constraints:
– Right of entry Could affect sampling
– Seasonality collection, quality, or
accuracy
– Physical location
Training, retirements,
– Available personnel knowledge retention
– Contracts
– Deadlines
14
S1: Intended Use of Data
• Decision Making vs. Estimation
– Confirmation of compliance in
regulatory limit
15
S1: Example
• Problem Description
– Historical data shows transformer spills in switchyard area but specific
locations and extent of condition is not known
• Team
– Electrical Project Manager (DM), Chemical Analyst (oils and PCBs), Field
Engineer, member from city health department (groundwater knowledge)
• Conceptual Model
– Most likely contaminated area is within 2 meters of the main transformer
stations and along the main drive route where components were shipped
in and out of the yard. Previous studies show the soil is sand/gravel and
thus potential for deep penetration. Rainfall is potentially moving PCBs
into the water table
• Resources and Constraints
– Deadline: within one year
– Sampling method available and approved
– Analysis laboratory will be shutdown for upgrades in 8 months
– Regulatory approval required to start sampling
– Budget available for team of 3 to conduct samples and analysis of < 75
samples
16
S2: Identify the Goal of the Study
• Identify principal study questions
• Consider alternative outcomes or
actions that can occur
• For decision problems, develop
decision statement(s), organize
multiple decisions
• For estimation problems, state what
needs to be estimated and key
assumptions
17
S2: Activities
• identify the principal study question and define
alternative actions that may be taken based upon the
range of possible outcomes that result from
answering the principal study question;
18
S2: Identify the Principal Study
Question
• Helps focus the search for
information to address the problem
• Examples: Decision problems
• Does the concentration of contaminants exceed
acceptable levels?
• Does a contaminant pose a human health or ecological
risk?
• Is the contaminant concentration significantly above
background levels?
20
S2: Developing Decision Statement
• Use principle question and alternate
actions to develop a set of decision
statements
• Determine if
(condition/issue/criteria) requires
(taking the following action)
21
S2: Developing an Estimation Statement
22
S2: Example
• Specify the primary question
– Does the concentration levels of PCBs in
the soil exceed the regulatory limit
• Determine alternative actions
– Remove soil to waste disposal site
– Decontaminate the soil to below clearance
levels
– Take no action
• Specify the decision statement
– Determine if the PCB contamination within
the soil requires disposal to waste
disposal site, soil simply cleaned to below
clearance levels
23
S3: Identify Information Inputs
• Identify and confirm:
– the types and potential sources of
information needed
– Information basis for specifying
performance or acceptance criteria;
and
– The availability of appropriate sampling
and analysis methods.
24
S3: Type of information needed
25
S3: Issues using existing data
• Often historical data or existing surveys are available
to provide information
• The quality and accuracy of the existing data may
challenge the outcome
– Confirm adequate quality assurance and information
control
– Confirm applicability of methods used and consistency
with methods chosen for the remainder of the data set
26
S3: Basis for Performance and
Acceptance Criteria
• Identify the basis for performance and acceptance criteria to guide steps 5
and 6
• For decision problems, the analytical approach requires decision rules
incorporating some type of Action Level, a threshold to determine choice of
alternative actions as per Step 2.
• The specific information source used to determine this level is done now.
Actual numerical value needed by Step 5.
• For action level relative to background, need to determine the basis for
characterizing background.
27
S3: Considerations for identifying and
evaluating appropriate sampling and
analysis methods
• Use list of environmental characteristics that are necessary
for addressing a particular decision or estimate
• Develop list of physical sampling and analytical methods
appropriate to obtain the necessary information (If no
methods, return to step 2 and change goal)
• Decision problems: bias kept to minimum. Mitigate:
– Non-representative sampling
– Instability or contamination of sample
– Interferences and matrix effects in analysis
– Calibration
28
S3: Example
• Type of information needed
– Historical data inaccurate. Need new information to be
obtained. Soil samples and water samples from nearby
runoff are to be taken to measure PCB levels.
• Source of information
– Data from existing pilot study will be used to identify
locations of soil samples.
• How action level will be determined
– Local regulations on PCB concentrations in water will
be used to determine the action level.
• Identify appropriate sampling and analysis methods
– A leachant method will be used for the soil samples to
extract the PCBs. PCB Aroclor analysis via gas
chromotagraphy will be used to represent PCB
concentration
29
S4: Define Boundaries of Study
• Define target population of interest and
relevant spatial boundaries
• Define what constitutes a sampling unit
• Specify temporal boundaries and other
constraints on sample/data collection
• Specific smallest unit on which decisions or
estimates are made
30
S4: Define Target Population
31
S4: Types of Spatial and Temporal
Boundaries
• Spatial boundaries define physical area and
general locations of samples
• Temporal boundaries describe time frame
and when samples to be taken
LATE SUMMER
32
S4: How to Determine Spatial
Boundaries
• Step 1 Conceptual Model provides input
• Define geographic area applicable for decision
making or estimation
– Entire area using latitude, longitude, altitude
(clear even if site is altered)
– Length, area, volume, legal boundaries
– Media, soil depth, water depth,
• Divide population into subset of relatively
homogeneous characteristics
– All same type of media (gravel, sand, concrete,
etc.)
– Subset has expected similar contamination level
33
S4: How to Determine Temporal
Boundaries
• Step 1 Conceptual Model provides input
• Consider temporal phenomena at site
– Weather variations,
– seasonal effects,
– equipment operation under different conditions
(temperature, moisture).
• When is decision or estimate relevant?
– May not be possible to collect data over full time
period the decision or estimate will apply
• Mainly future use applications (i.e. residual dose
to resident…)
34
S4: Constraints on Data Collection
• Access to Property
• Availability and operation of equipment
• Environmental Conditions when sampling is not
possible (high humidity, freezing temperatures)
• Availability of personnel
– Sampling team, QA/QC team, inspectors, regulators
35
S4: Define Scale of Inference
36
S4: Additional Guidance on Scale
37
S4: Example
• Target Population
– Target population consists of all possible samples of PCB
concentrations in soil and in water.
• Spatial/Temporal boundaries and Constraints
– The spatial boundary consists of the following subsets:
general switchyard, transformer location, vehicle travel
route, and nearby drainage ditch for water samples.
– Temporal boundaries will include high water run off and
low water run off times.
– No measurements in winter as ground is frozen which may
bias results.
– Environmental Inspector required for at least one
sampling period.
• Scale of Inference
– Higher sample density in transformer subset.
38
S5: Develop the Analytical Approach
40
S5: Decision – Action Levels
41
S5: Action level – detection limits
42
S5: Develop Theoretical Decision Rule
• “If…Then…Else…”
Example:
The study will estimate the total annual concentrations
of mercury in the effluent from the facility within 500
meters of the discharge.
44
S5: Example
• Specify Appropriate Action Level
– The clearance level for Cs-137 is 0.1
Bq/g. Action level is set to clearance
level.
• Specify Theoretical Decision Rule
– If the mean concentration of Cs-137 in
the soil exceeds 0.1 Bq/g, then the soil
shall be remediated, else the soil is
clear for release.
45
S6: Specify Performance or
Acceptance Criteria
• For decision problems, specify the
decision rule as a statistical hypothesis
test, examine consequences of making
incorrect decisions and place acceptable
limits on the likelihood of making
decision errors.
47
Total Study Error
(Total Variability)
ERROR
SOURCES Measurement Error
Sampling Error
(Measurement
(Field Variability)
Variability)
Sample
Packaging Subsampling
Delineation
Sample
Transport Extraction
Extraction
Analytical
Storage
Determination
49
S6a: Possibility of incorrect decisions
50
Action
Level
False (β)
Acceptance
False
Rejection
(α)
51
S6a: Defining Baseline Condition
52
Gray
Region
Action
Level
53
S6a: Establishing Tolerable Limits on
Decision Error Probabilities
• False Rejection decision error limit (α)
• False Acceptance decision error limit (β)
– Most stringent error is usually 1% or 0.01
– Use as starting point and consider consequences of
error
– If consequences are low, possible to increase the
allowable error limit
Example: Action Level = 100; (α) = (β) = 0.01. Gray Region narrow.
Consequence of False Rejection is increased disposal to waste site.
Conservative.
Consequence of False Acceptance is increased exposure to
environment. Consequence at parameter value 100-120 is low.
Consequence at parameter level >120 moderate. Gray region could
be extended up to 120 and allow for a higher probability of error.
54
S6b: Estimation
• Mean
• Median
• Upper percentile, upper confidence limit
• Slopes
• Ratios
• Rates of Processes
• Estimates of Toxicity
• Spatial Contours
• Population size, growth rate, decay rate
55
Median Mean
Action Level
-1 Sigma +1 Sigma
56
S6b: Uncertainty
57
S6b: Standard Error
• Function of:
– Amount of data available
– Underlying distribution (normal or not)
– Variability in the data
58
S6b: Intervals
• Confidence Intervals
– Descries population mean and standard
deviation
• Tolerance Intervals
– Contain the population that has a
significant probability
– Useful for non-normal distributions
• Prediction Intervals
– Generated from the data to predict a
future mean and standard deviation
59
S6: Example
• Setting baseline and alternative conditions
– Will use a statistical hypothesis test. PCB in soil guidelines give 0.5
mg/kg and 1.3 mg/kg for agricultural and residential/parkland use
respectively.
– Baseline condition is soil is not hazardous as PCB concentrations will be
less than 0.5 mg/kg. Alternative condition is that the site exceeds
allowable concentrations and is hazardous.
• Determine impact of decision errors
– False acceptance – actually hazardous. Consequence is that users of end
state will have higher health risk due to higher levels of PCB
concentrations in the soil.
– False Rejection – actually not hazardous. Additional remediation costs
will incur.
• Specify the ‘gray region’ for the Decision Performance Curve
– Gray region extended to 0.7 mg/kg. Guidelines do not show significant
risk at these values and end use is not necessarily targeted for
agricultural application.
• Set the tolerable decision error limits
– False acceptance decision error limit of 0.05 (5%)
– False rejection decision error limit of 0.15 (15%)
60
S7: Develop Detailed Plan for Obtaining
Data
• Compile all information and outputs
from Steps 1 through 6
• Use information to identify
alternative sampling and analysis
designs appropriate for use
• Select and document a sampling
design that will yield data to achieve
performance or acceptance criteria
More detail can be found in EPA QA/G-5S and EPA QA/G-9S
61
S7: Types of Information Required
• Your objectives and intended use of the data
• Outputs from Steps 1 through 6 of the DQO
Process
• Background information on the problem
– Site properties, technical characteristics of
contaminants/media, regulatory requirements,
known spatial/temporal patterns
• Expected variability for the data based upon
similar studies
• Preliminary information on the underlying
distribution of the data that may impact
calculations
62
S7: Two Types of Sampling Design
63
S7: Factors affect Sampling Designs
64
S7: Factors affect Sampling Designs
65
S7: Representation
• The extent to which measurements actually
reflect sampling unit.
• Degree to which samples represent the
population.
66
Total Population
Mean: 54
σ = 26 Action Level
100
Strata 1 Population
Mean: 49
σ = 23
67
S7: Determine Amount of Data
68
S7: Revisiting the Plan
• If the sampling plan does not produce
the desired result, it can be adjusted
by considering the following:
– Increase the tolerable limits on decision
errors
– Increase the funding for sampling and
analysis
– Change the boundaries of the study
– Relax project constraints
69
S7: Requirements documenting
the sampling plan
• Design elements include:
– Number of samples,
– Sample type (composite vs. individual),
– General collection techniques,
– Physical sample (Amount of material to be collected),
– Sample support (Area, quantity that each individual sample
represents),
– Sample locations (surface coordinates and depth) and how
selected,
– Timing considerations of collection, handling, and analysis,
– Analytical methods, and
– Statitical sampling scheme
70
Iterations may be required
DQO
DQA
From EPA-QA/G4
71
What did we learn?
72