Professional Documents
Culture Documents
Abstract
In this paper, we expand upon established uncertainty analysis techniques to demonstrate a general method for
assessing variability and uncertainty in Gaussian air pollutant dispersion modeling systems. To illustrate this method,
we estimated variability and uncertainty in predicted hexavalent chromium concentrations generated by welding
operations at a shipbuilding and repair facility in California. Using Monte Carlo statistical techniques, we propagated
uncertainty across both ISCST3 and AERMOD, and estimated the contribution of variability and uncertainty from
four model components: emissions, spatial and temporal allocation of emissions, model parameters, and meteorology.
Our results indicated the 95% confidence interval uncertainty at each receptor spanned an order of magnitude. From a
practical perspective uncertainty is most important at receptors with highest predicted concentrations. In this case
study, emissions were the primary source of uncertainty. However, Gaussian models are also sensitive to location of
emission releases, meteorology, and model parameters. Simplified modeling approaches may lead to errors in pollutant
concentration estimates, especially in close proximity to emissions sources where predicted concentrations are highest.
r 2003 Elsevier Ltd. All rights reserved.
Keywords: AERMOD; Air quality modeling; Air toxics; Emissions inventory; Hazardous air pollutants; Hexavalent chromium;
ISCST3; Uncertainty analysis; Welding
1352-2310/03/$ - see front matter r 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S1352-2310(03)00411-4
ARTICLE IN PRESS
3482 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
annual process activity rates. We quantified annual content, including estimating a range of the ratio of
process rates by process-alloy combination using filler CrVI in total generated fume, and a range of the ratio of
metal codes specified by the American Welding Society CrVI to total chromium in fume. We applied the same
(AWS) (AWS, 1969). To determine whether these three assumptions to both CrVI fume content estimates and
years represented longer-term trends at the facility we fume generation rates.
reviewed data with facility engineers. Based on these Efron and Tibshirani (1993) describe how to estimate
discussions we determined nearly all filler metals are the bootstrap standard error of a population statistic,
used in the year they are purchased. We also estimated which we applied to estimate the distribution of annual
that because of uncertainty about future on-site activity average emissions estimates from the facility. We used
with respect to each process-alloy combination, alloy polynomial regression techniques to fit process rate and
usage by process might vary by as much as one-third emission factor data to cumulative probability
higher than the highest observed value or one-third density functions (PDFs). The use of polynomial
lower than the lowest observed value. regression is analogous to using a continuous empirical
To assess uncertainty in fume generation rates we distribution as described by Law and Kelton (2000). To
assessed peer-reviewed studies, many of which were ensure proper fit, we accepted regressions with R2
summarized for the US Environmental Protection values exceeding 0.95 whose behavior did not result in
Agency (MRI, 1994). We extracted fume generation negative process rate or emission factor values. Emis-
rate data from each study and categorized them by sions were propagated individually by process-alloy
process-alloy combination. Several data sources devel- combination.
oped by the AWS indicated process factors like Table 1 demonstrates the distribution of possible
voltage and current affect emission rates. Other data in average annual CrVI emission rates by process-alloy
the literature did not confirm these results, probably combination. Our analysis suggested the 95% con-
because source testing and analytical methods fidence interval of predicted CrVI emissions reached an
varied between studies, and so we did not consider order of magnitude in many cases. Emissions from the
dependence between these factors and emission rates. GFCAW process and SMAW-low alloy combination
Only limited data representing the GFCAW process and exceeded one order of magnitude. This uncertainty arose
all processes involving nickel alloys were available. We from the lack of sufficient data available to analyze CrVI
assumed the GFCAW process is similar to either the content in fume for these processes. Over 98% of all
GMAW or SMAW process and processes involving alloys processed at the facility were mild and low alloy
nickel alloys behave similarly to processes involving steels, and we estimated the use of these alloys
stainless steels. accounted for 60% of CrVI emissions. Emission factors
Three types of data are available to estimate the for mild and low alloy steels are generally two orders of
amount of CrVI in fume: direct emission factor studies, magnitude lower than for stainless steels.
industrial hygiene studies (Castner and Null, 1998), and
AWS standards for consumable electrode composition.
All studies provided information about CrVI generated 4. Uncertainty due to spatial and temporal allocation of
as a function of alloy type, but fewer studies provided emissions
electrode classifications sufficient for determining initial
chromium content. Because industrial hygiene and The case study facility generally operates 16 h per day,
emission factor data are not directly comparable, we 6 days per week excluding Sundays. Because welding
used several different techniques to estimate fume CrVI activity is constant during operating hours, we did not
Table 1
Distribution of CrVI emissions by process/alloy combination
2.5 25 50 75 97.5
estimate a range of uncertainty or variability in the any information to estimate the characteristics of this
temporal allocation of emissions. In this case study the variability and uncertainty, we applied uniform distribu-
spatial location of emissions was uncertain because tions (Law and Kelton, 2000). In our simulation the
welding equipment is mobile and locations cannot be location of emissions fluctuated between multiple
precisely identified, and variable because activities in any volume sources in each of the six volume source groups,
given day are based upon the type of work scheduled for and between each of the 13 generalized locations. When
that day at various locations. conducting Monte Carlo analysis, we selected a single
To estimate the spatial allocation of emissions, we emission value by process alloy combination for each
toured the facility and discussed the location of welding iteration. We then allocated emissions by sampling from
operations with facility engineers and managers. uniform distributions representing the ratio of emissions
Through our discussions with facility personnel we from each process-alloy combination attributable to
determined certain types of welding process-alloy each release location. When allocating emissions from
combinations can be identified to occur only in specified each process-alloy combination to each location we
areas. These judgements, while potentially subject to ensured the total mass of emissions was conserved. Our
bias and error (Morgan and Henrion, 1990), are based analysis indicated emissions at most locations fluctuated
upon the best information available regarding the 20–30% in either direction of median emission rates.
subject facility. We identified 13 generalized locations Predicted variability and uncertainty in the allocation of
where welding can occur, as shown in Fig. 1. We used six emissions to the machine shop was minimal (o5% in
area sources to represent emissions generated on either direction of median emission rates) because nearly
automated manufacturing lines where most GFCAW all stainless steel emissions were released from this
welding is conducted, a volume source to estimate location.
emissions from a machine shop where most stainless
steel welding was conducted, and six groups of volume
sources to estimate the location of welding emissions 5. Uncertainty due to model parameters
generated from outdoor machining areas, dry docks,
and wet docks. Both ISCST3 and AERMOD can be run with some
Based on discussions with facility personnel we degree of variation in model setup consisting of
assigned a range of the percent of total emissions for differences in model parameters (US EPA, 1995, 1998).
each process-alloy combination to each source. Lacking Our approach is to assign a single model parameter
Fig. 1. Schematic map of source locations for three different approaches: database, health risk assessment (HRA), and Monte Carlo
(MC).
ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489 3485
where one value is appropriate, and to assign a range of 6. Uncertainty due to meteorology
values where one would not be appropriate. When a
range is assigned, we conduct a sensitivity analysis in Weather conditions and meteorological data sets used
each model to determine the range of predicted to model them exhibit variability over time. The
concentrations at each receptor that can be generated application of meteorological data sets from a single
from the range of model parameters. The distribution of location to a modeling domain is also uncertain. In this
predicted concentrations represents parameter uncer- study, we used on-site meteorology and considered how
tainty. inter-annual variability in meteorology affects uncer-
For this case study, all modeling runs were conducted tainty in long-term annual average CrVI concentration
using the urban option in ISCST3 and AERMOD. We estimates. Two data sources were available for use: a
did not consider atmospheric reduction or deposition of proximate National Weather Service site and a tempor-
CrVI due to the proximity of receptors to sources. ary monitoring site located in the modeling domain.
Model parameter ranges were assigned for release Observations at the temporary monitoring site were
parameters at each location, terrain, building down- measured about 1 km downwind of emission sources, as
wash, albedo, Bowen ratio, and surface roughness. shown in Fig. 2. To develop a meteorological data set
To run each model, we specified a base case consisting suitable for Gaussian models we combined on-site
of median emission rates from each release location, a surface measurements with upper air conditions from
single meteorology data file, and a single choice for each the nearest NWS site, located about 3 miles northwest of
model parameter. To estimate uncertainty caused by the case study domain. Meteorological characteristics
parameter configurations, we ran the base case multiple between the case study site and NWS site were generally
times for each model, replacing the base case model similar, but some differences between the two sites were
parameters with different parameter configurations. For apparent, and were caused by differences in local
this case study, we analyzed four receptors located geographic and topographic conditions. We used the
approximately 100 (receptor 1), 500 (receptor 2), 1000 on-site data set as our base case for modeling, and
(receptor 3), and 1500 (receptor 4) meters downwind of assumed that the data set adequately characterized
emission sources, as shown in Fig. 2. Modeling results meteorological conditions in the case study domain.
suggested predicted concentrations ranged about 30% in To analyze inter-annual variability in meteorology, we
either direction of median values due to parameter first obtained 10 years of surface and upper air
uncertainty. observations at the nearest NWS site. We then ran both
Fig. 2. Map of the modeling domain showing source and receptor locations. Contour lines indicate median annual average
concentrations (ng m3) of CrVI as predicted by ISCST3 using on-site meteorological data.
ARTICLE IN PRESS
3486 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
models in the base case, replacing the on-site meteor- NWS data and assumed variability at the NWS site
ological data set with each of ten data sets based upon would be equivalent to variability at the on-site location.
NWS data, and analyzed concentration results at each We analyzed this variability by normalizing these results
of the four analyzed receptors. Results indicated up to to predicted concentrations at each receptor and
25% variability in pollutant concentrations at each calculating a standard deviation. We then applied this
receptor might be expected due to inter-annual varia- standard deviation to the base case model results at each
bility in meteorology. receptor.
After expressing variability and uncertainty in model
components to PDFs, we defined an equation to post-
process these distributions and estimate a range of
7. Monte Carlo analysis to estimate model uncertainty
pollutant concentration results.
To estimate a range of predicted pollutant concentra- Pc ¼ EM SA Cr Met MP; ð1Þ
tions we ran both ISC and AERMOD in base case
configurations. Running Gaussian models with unit where, Pc is the annual average pollutant concentration
emission rates for each of the source-receptor combina- estimate (ng m3), EM the annual average emissions by
tions is equivalent to running each model with specified process/alloy combination (kg yr1), SA the Spatial
emission rates. From model results, we developed a allocation by process/alloy combination and release
matrix of predicted unit concentrations generated by location (percent deviation from base case), Cr the Base
each source at each of the four analyzed receptors. case unit concentration at each receptor (ng m3), based
To propagate variability and uncertainty, data repre- upon unit emission rate from each location, base case
senting each model component were fit to PDFs using model parameters, and on-site meteorology, Met the
polynomial regression unless otherwise indicated. We inter-annual variability in meteorology (percent devia-
compiled emissions distributions by process-alloy com- tion from base case) and MP the Uncertainty in release
bination, and spatial allocation distributions by process- parameters (percent deviation from base case).
alloy combination and release location. Uncertainty in To propagate uncertainty using Eq. (1), we applied a
model parameters was normalized to predicted concen- simple random sampling Monte Carlo technique de-
trations at each receptor using the base case. To assess scribed by Efron and Tibshirani (1993). It is important
variability due to meteorology, we analyzed 10 years of to note that any equation used to propagate variability
Fig. 3. Distributions of uncertainty for each of the four receptors simulated by Monte Carlo runs. Distributions are represented by box
and whisker plot: 212 percentile, 10th percentile, 25th percentile, mean, 75th percentile, 90th percentile, and 9712 percentile.
ARTICLE IN PRESS
T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489 3487
and uncertainty in a Monte Carlo approach requires to the predicted base case pollutant concentration.
independence between each of the components (Frey Emissions were the dominant source of uncertainty in
and Rhodes, 1999). We believe the assumption of this case. Both the spatial allocation of emissions and
independence is appropriate for Gaussian models, but meteorology would have been more important factors in
may not be appropriate for more complex modeling this case if information specific to the facility and
systems that incorporate photochemistry. modeling domain had not been obtained. Fig. 4 also
demonstrates AERMOD is capable of producing a
greater range of results than ISCST3, and this range is
inversely proportional to source-receptor distance for
8. Results low-level sources.
Fig. 4. Distributions of uncertainty at each of the modeling receptors for five model components: emissions inventory (Emis), spatial
allocation of emissions (SA), dispersion models (ISCST3 and AERMOD), and Meteorology (Met). (A) Receptor 1, (B) Receptor 2, (C)
Receptor 3, (D) Receptor 4. Distributions are represented by box and whisker plot: 212 percentile, 10th percentile, 25th percentile, mean,
75th percentile, 90th percentile, and 9712 percentile. All distributions are normalized to the predicted base case concentration at each
receptor.
ARTICLE IN PRESS
3488 T. Sax, V. Isakov / Atmospheric Environment 37 (2003) 3481–3489
Table 2
Comparison of Monte Carlo analysis results to simplified modeling approaches
Receptor Median Monte 95% confidence interval Regulatory health risk Database approach
Carlo result Monte Carlo results assessment approach
Models, Vol. 1. Theory and Methodology Based Upon Law, A.M., Kelton, D.W., 2000. Simulation Modeling and
Bootstrap Simulation. US Department of Energy DOE/ER/ Analysis, 3rd Edition. McGraw-Hill, San Francisco, CA.
30250—Vol. 1. Midwest Research Institute (MRI), 1994. Development of
Frey, H.C., Zheng, J., 2002. Method for Development of Particulate and Hazardous Emission Factors for Electric
Probabilistic Emission Inventories: Example Case Study for Arc Welding (AP-42 Section 12.19) Final Report. EPA
Utility NOx Emissions. Emissions Inventories—Partnering Contract No. 68-D2-0159, US Environmental Protection
for the Future. Proceedings of the EPA 11th International Agency.
Emission Inventory Conference, April 15–18, 2002. Avail- Morgan, M.G., Henrion, M., 1990. Uncertainty. Cambridge
able at http://www.epa.gov/ttn/chief/conference/ei11. University Press, New York.
Hanna, S.R., Chang, J.C., Fernau, M.E., 1998. Monte Carlo Seigneur, C., Constantinou, E., Permutt, T., 1992. Uncertainty
estimates of uncertainties in predictions by a photochemical Analysis of Health Risk Estimates. Document No. 2460-
grid model (UAM-IV) due to uncertainties in input 009-510, Electric Power Research Institute, Palo Alto, CA.
variables. Atmospheric Environment 32, 3619–3628. US Environmental Protection Agency (USEPA), 1995. User’s
Hanna, S.R., Zhigang, L., Frey, H.C., Wheeler, N., Vukovich, Guide for the Industrial Source Complex (ISC3) Dispersion
J., Arunachalam, S., Fernau, M., Hansen, D.A., 2000. Model. Volume I. EPA-454/B-95-003a, USEPA, Research
Uncertainties in predicted ozone concentrations due to Triangle Park, NC 27711.
input uncertainties for the UAM-V photochemical grid US Environmental Protection Agency (USEPA), 1998. User’s
model applied to the July 1995 OTAG domain. Atmo- Guide for the AMS/EPA Regulatory Model—AERMOD.
spheric Environment 35, 891–903. A-99-05, II-A-2, OAQPS, Research Triangle Park, NC
Irwin, J.S., Rao, S.T., Petersen, W.B., Turner, D.B., 1987. 27711.
Relating error bounds for maximum concentration esti- Venkatram, A., 1981. Model predictability with reference to
mates to diffusion meteorology uncertainty. Atmospheric concentrations associated with point sources. Atmospheric
Environment 21, 1927–1937. Environment 15, 1517–1522.