Uncertainty Analysis

for
Flow Measurements and Techniques
using
Standardized Methodology
Overview

 Background
 Uncertainty Analysis (UA) Frameworks
 AIAA (1995)
 UA Implementation Example
 Methodology
 Assessment of Elemental Uncertainties
 Customized GUI for UA Implementation
 Conclusions
 Outlook
 Background
SAMPLE REQUEST
regarding uncertainty analysis originated from a Hydrologic Service

…. Has anybody out there had to defend the validity of an ADCP flow
measurement against a legal challenge from a third party?
……When current meters were used to undertake these measurements we could
claim that the flow measurement was undertaken in conformance with British
and International standards for current meter gauging and that the current meter
had a valid calibration certificate…
In the case where flow measurements are now taken using ADCPs we feel more
vulnerable to legal challenges. This is for two reasons:
1. There is no ISO document in place. The Agency has to rely on its own internal
document on gauging procedures which is based on the draft ISO document.
2. ADCPs do not have "certificates of calibration" . The only checks on the
performance that can be made are against other ADCPs or other types of flow
monitoring equipment.

(posted on the USGS’ Hydro-Acoustics Work Group webpage by

R. Iredale, The Environment Agency of England and Wales, 2005)
 UA Frameworks
Over the last 50 years, considerable efforts have been made by professional societies
to develop and implement uncertainty analysis (UA).
One of the rigorous UA methodology (based on sound statistical and engineering
concepts):
Guide to the Expression of Uncertainty Measurement (ISO, 1993)
- adopted widely by various scientific & research communities, e.g., NIST (1994), NF
ENV 13005 (1999)
- the guide recognizes the need for further adaptation for specific areas

Specific adaptations for engineering:

- Assessment of Wind Tunnel Data Uncertainty (AIAA, 1995)
- Test Uncertainty (ASME, 1998)

Key assumptions/concepts for ISO (1993)-based standards

- Gaussian pdf-s for the error sources
- 2 sample standard deviations for 95% confidence level
- for large samples (N ≥ 10), special procedures for handling small samples
- RSS used for combining uncertainties
- Taylor-series expansion for propagation of uncertainties
- total uncertainties expressed using confidence intervals
 UA Frameworks
Terminology for ISO (1993) - based standards
ISO (1993) AIAA-S-071-1995
Uncertainty of a measurement
 input quantity  individual variable
 type A standard uncertainty  bias limit
 type B standard uncertainty  precision limit (differently estimated
 combined standard uncertainty for single and multiple tests)
 total uncertainty
Uncertainty of a result
 functional relationship  data reduction equation
 bias limit
 precision limit (differently estimated
for single test with single readings
and averaged readings, multiple tests)
 sensitivity coefficients  sensitivity coefficients
 combined standard uncertainty  combined standard uncertainty
(accounts for correlated errors) (accounts for correlated errors)
 expanded uncertainty (accounts  uncertainty at specified confidence
for the level of confidence) level
 coverage factor f( degrees of  coverage factor f( degrees of freedom
freedom and t Distribution) and t Distribution)
The 3 standards provide the same total measurement uncertainty
ASME PTC 19.1-1998
 independent parameter
 systematic uncertainty
 random uncertainty (differently
estimated single and multiple tests)
 measurement uncertainty
derived result
 systematic uncertainty
random standard deviation
(differently estimated for single
and multiple tests)
 sensitivity coefficients
 uncertainty of the result (accounts
for correlated errors and the level
of confidence)
 coverage factor f(degrees of
freedom and t Distribution)
 AIAA (1995)
Engineering approach, simple, clear, widely applied

 = total error
  = bias error
 = precision error
 k+ 1

k k k+ 1
X
 Bias error (b): fixed, systematic X tru e X k X k+ 1

(a) tw o read in g s
 Bias limit (B): estimate of b

 Precision error (e): random

FREQ UENCY O F O CCURRENCE

 

 Precision limit (P): estimate of e

 Total error: d = b + e

X
X tru e 
M A G N ITU D E O F X
(b ) in fin ite n u m b e r o f read in g s
 AIAA (1995)

Implementation Sequence
Determine the data reduction equation
Key feature: data-reduction equation r  r X1, X2 ,, X J 
r = r(X1, X2, X3,…, Xj)

Identify sources of uncertainty for each X i

E LEM ENTA L
E RROR S O UR CES

IND IVIDUA L
1 2 J M E AS URE M E NT
S YSTEM S Assess relative significance of uncertainty
sources (order of magnitude estimates)

X X X M E AS URE M E NT
1 2 J O F IND IV IDUAL
B ,P B ,P B ,P VAR IAB LE S
1 1 2 2 J J

Considering the significant sources, estimate the

precision and bias limits for each X i
DATA RE DUCT ION
r = r (X , X ,......, X ) E QUAT IO N
1 2 J

r E XPER IM EN TAL
For the experimental result r, determine the
B,P
r r
RE SU LT precision and bias limits and overall uncertainty
 AIAA (1995)
Implementation Aspects
 Measurement systems for each individual variable Xi : instrument, data acquisition and
reduction procedures, operational environment (laboratory, in situ), the flow and its
interaction with the instrument and the environment
 Estimates of errors are meaningful only when considered in the context of the process
leading to the value of the quantity under consideration
 Uncertainties estimated following the signal propagation from sensor to the
final result
 Uncertainties estimated with a pre-established confidence level (95% for most
engineering areas)
 UA differently conducted dependent on the type of experiment:
 Single test (for complex or expensive experiments): one set of
measurements (X1, X2, …, Xj) for r

 Multiple tests (ideal situations): many sets of measurements (X1, X2, …,

Xj) for r at a fixed test condition with the same measurement system
 AIAA (1995)
MULTIPLE TESTS (recommended)

Given a data reduction equation for a measurement r  r ( X 1 , X 2 ,, X J )

1 M
The result and its uncertainty is r Ur with r
M
r
k 1
k

The uncertainty in the final result U r = B 2r + P 2r  1 / 2

where the bias limit of the result is

J J 1 J r
B    B  2
2
r i
2
i
2
  i k Bik i 
i 1 i 1 k i 1 X i

and the precision limit of the result is

1/ 2
tS r  M rk  r 2 
Pr  ; S r    ; t  2 for M  10
M  k 1 M  1 
 AIAA (1995)
SINGLE TEST

Given a data reduction equation for a measurement r  r ( X 1 , X 2 , X 3 , X j , )

The result and its uncertainty is r Ur

The uncertainty in the final result U r = B 2r + P 2r  1 / 2

where the bias limit of the result is

J J 1 J
r
B    B  2
2
r i
2
i
2
  i k Bik i 
i 1 i 1 k i 1 X i

and the precision limit of the result is

1/ 2
 r  r  
M 2

Pr  tS r S r   k   Based on prior information

 k 1 M  1 
 AIAA (1995)

Implementation Aspects

 sound engineering judgment to optimize the output with minimum costs, e.g.:
 use of end-to-end uncertainty estimation approach
 uncertainty sources < 1/4 or 1/5 of the largest sources are usually considered
negligible

 specific procedures for single and multiple measurements

 specific procedures for dealing with small statistical samples

 methodology for assessment of calibration uncertainties

 methodology for data validation

 D E F IN E P U R P O S E O F T E S T A N D
R E S U LT S U N C E R TA IN T Y R E Q U IR E M E N T S

AIAA (1995) S E LE C T U N C E R TA IN T Y M E T H O D

D E S IG N T H E T E S T
- D E S IR E D PA R A M E T E R S (C D , C R,....)
- M O D E L C O N F IG U R AT IO N S (S )
- T E S T T E C H N IQ U E (S )
Implementation -
-
M E A S U R E M E N T S R E Q U IR E D
S P E C IF IC IN S T R U M E N TAT IO N
- C O R R E C T IO N S T O B E A P P L IE D
Aspects
D E T E R M IN E E R R O R S O U R C E S
A F F E C T IN G R E S U LT S

 Integration of UA in all YES

phases of the E S T IM AT E E F F E C T O F
T H E E R R O R S O N R E S U LT S

measurement
NO U N C E R TA IN T Y NO
IM P R O V E M E N T
A C C E P TA B L E ?
P O S S IB L E ?

NO YES

NO TEST IM P L E M E N T T E S T

S TA R T T E S T

M E A SU R E-
R E S U LT S NO MENT
A C C E P TA B L E ? SYSTEM
P R O BL E M ?

YES YES

NO C O N T IN U E T E S T S O LV E P R O B L E M

D O C U M E N T R E S U LT S
YES E S T IM AT E - R E F E R E N C E C O N D IT IO N
PU RPO SE
A C T U A L D ATA - P R E C IS IO N L IM IT
A C H IE V E D ?
U N C E R TA IN T Y - B IA S L IM IT
- T O TA L U N C E R TA IN T Y
 AIAA (1995)

IMPLEMENTATION EXAMPLE
 Extensively used in laboratory measurements and field conditions,
from simple (Pitot tube) to complex (LDV) instruments

 Widely applied for teaching and research purposes

 Successful implementation to discharge measurements:

 conventional instruments (Muste et al. 2007)
 contemporary, nonintrusive techniques:
 Large-Scale Particle Image Velocimetry (Y-S. Kim et al, 2007)
 Acoustic-Doppler Current Profilers (Gonzalez-Castro & Muste, 2007)
 ADCP UA: Implementation
 Currently, ADCPs are the most efficient instrument for riverine
environment characterization (monitoring and research needs)

 If properly operated, the instrument can accurately document

discharges, mean velocities, and selected turbulence characteristics

 Despite their extensive use, there are aspects regarding their

capabilities, operation, and uncertainty analysis not documented yet
 ADCP UA: Implementation

ADCP Uncertainty Analysis (UA) status

Past efforts (non-standardized methodologies)

 Discharge: Simpson & Oltman (1992), Gordon (1993),
Lipscomb (1995), Morlock (1996), Simpson (2001), Gartner
(2002), Muller (2002), Yorke & Oberg (2002), USGS-RDI (2005)
 Turbulence measurements: Droz (1998), Stacey (1999),
Nystrom (2002), Schemper & Admiraal (2002)

On-going efforts (standardized methodology)

 UA formulated within the framework of authoritative
engineering standards
ADCP UA: Implementation
Discharge Measurement with ADCP mounted on a boat

Unmeasurable Near-bank Areas

Unmeasurable Top Area Qet

Qel Qer

Qm
Measurable Area

Qeb
Unmeasurable Bottom Area

Qt  Qm  Qem  Qet  Qeb  Qel  Qer

where Qel  KVl Ll Z l ; Qer  KVr Lr Z r

 ADCP UA: Elemental Uncertainty Assessment
Error identification
Source
e1: Spatial resolution
e2: Doppler noise
e3 : Velocity ambiguity
e4 : Side-lobe interference
e5: Temporal resolution
e6: Sound speed
e7 : Beam angle
e8 : Boat speed
Vf e9: Sampling time
ADCP9
e10 : Near-transducer
e11: Reference boat velocity
e12: Depth
e13: Cell positioning
e14: Rotation
e15: Timing
e16: Edge
e17: Vertical profile model
e18: Discharge model
e19: Finite summation
e20: Site conditions & operation
Accounted in Reduction
Biases Estimation of Depends upon Can be estimated from
Equations through 1
Water and boat velocities, depths † ADCP, mode, settings, boat speed End-to-end calibration 2
ADCP frequency, mode, settings, UA of signal processing algorithms,
Water and boat velocities Bva, Bvb
speed of sound, gating time instrument intercomparison
Water and boat velocities † Mode, settings End-to-end calibration
Discharge through unmeasured areas * Beam angle, settings, bathymetry End-to-end calibration
High frequency velocity components † Settings End-to-end calibration
UA of C(Salinity, Temperature) with data
Water and boat velocities, depths BC Water properties
from reference meter
Water and boat velocities, depths B ADCP Manufacturer’s specifications
Water and boat velocities, depths † Site, flow, boat operation End-to-end calibration
Frequency of large-scale flow Instrument intercomparison based on long
Long-term means †
structures 3 data records under steady conditions
ADCP, draft, settings, velocity, flow Experimental Measurements and CFD
Velocities near the ADCP Bnt
depth Modeling
Water and boat velocities, depths Bvb Sediment concentration, flow 4 Manufacturer’s Specifications
UA of depths as f(C and gating time) and
ADCP, settings, draft, bathymetry,
Discharge through unmeasured areas BDa, BDp, BDp, BDo, BDavg 5 BC, Bt and BDADCP and concurrent depth
water properties, time gating
range measurements
Measured and unmeasured discharge Bt , BDa, BDp, BDo, BDavg ADCP, setting, water properties 
Water and boat velocities, depths
Bp , Br , Bh ADCP, setup, site Manufacturer’s Specifications
and geographic orientation
Distances by gating and discharge Bt ADCP, speed of sound, gating time Manufacturer’s Specifications
ADCP settings, bathymetry, cross
Discharges through channel edges B ,BL Manufacturer’s Specifications
section, edge distances
Discharge through unmeasured top Velocity distribution model, turbulence Field and Laboratory Experiments with
BQ1 6
and bottom areas intensity reliable CFD-LES Modeling
Highly resolved data / End-to-end
Discharge through measured area BQ2 6 Discharge model
calibration
6
Discharge through measured area BQ3 ADCP settings, boat velocity 
Total discharge † Site, boat operation Concurrently measured data
ADCP UA: Implementation
Data Reduction Equations (Teledyne/RDI’s ADCP)


V f  u f , v f 


Vb  u b , vb 
y

T zU ( t )
Qm     V
0 zL (t )
f  
 V b  k dz dt
 ADCP UA: Implementation

Exact approach – discharge in the direct measured area

Qi , j

i 1 i i 1 i  Ns

j
j 1
unit area for Qi , j

 Using BT

N s 1 n
z  z j t i 1  t i 
Qm  
i 1 j 1
j 1

2 sin 
vb 4  vb3 v1  v 2  vb1  vb 2 v4  v3 i 1, j
 ADCP UA: Implementation
Exact approach: in-bin discharge
 Using BT

Qmi , j  u a vb  ub va i 1, j z j 1  z j ti 1  ti 

WATER VELOCITY WITH RESPECT TO ADCP

1
ua  2 cos R csc  v1  v2  + sin Rcos P sec  v1  v 2  v3  v4   2 sin P csc  v 4  v3 
4
1
va  2 cos P csc  v4  v3  - sin P sec  v1  v2  v3  v4 
4
BOAT VELOCITY WITH RESPECT TO CHANNEL BED
1
ub   2 cos R csc  vb1  vb 2  + sin Rcos P sec  vb1  vb 2  vb3  vb 4   2 sin P csc  vb 4  vb3 
4
1
vb   2 cos P csc  vb 4  vb3  - sin P sec  vb1  vb 2  vb3  vb 4 
4
ACTUAL WATER VELOCITY
C
If v  FD , the discharge
V f isV aa V
functional relationship
b  u a  u b , v a  vb  of the form:
2 FS

 DEPTHS
Qmi , j  f FD1 , FD 2 , FD 3 , FD 4 , FS ,DC ,FDD b1, DFD;b1 , FD b1 , FD b1 ,  , r , p, h,  t a ,  z
top ADCP B

D p  D0  D a D p  Da
D B  Db  for Mode 1; D B  Db  for Mode 5
 ADCP UA: Implementation
Exact approach – top and bottom discharges (extrapolation)
DIS CHARGE (m 3 /s )
Z3
D ADCP
TOP LAYER 3-P OINT SLOP E
TOP Q (ES TIMATED) P OWER
DB CONS TANT
Z2
D top ADCP
TRANSDUCER FACE ACTUAL
SCALAR P ROFILE

D TRIPLE (m 2/s 2 )
PRODUCT
DEPTH D total
CELL Da
MID Q ADCP
D avg MEASURED ADCP
DISCHARGE VELOCITIES

Z P OWER FIT

D LG
Z1 P OWER
BOTTOM LAYER
BTM Q (ES TIMATED) P OWER IN LOW 0.2 D total

CONS TANT

m

Da Z 3 b 1
 Z2b 1
t i 1  t i  u a vb  ub va  j Da Z b 1
1
m
ti 1  t i  u a vb  ub va j
j 1
Qbi  Qt 
j 1

Z b 1
2
 Z1 b 1
 i
Z b 1
2
 Z b 1 1

 ADCP UA: Implementation
Uncertainty Propagation to Final Result: Bias Limit
BQ2  BQ2  BQ2  BQ2  BQ2  BQ2  BQ2
t m em et eb el er

   Q  2 2 2
 Qmi , j  2  Qmi , j  2 4   Qmi , j
2
 2  Qmi , j  2   
2

  
 n   p  p  r  r     
mi , j
  B 2
 B   B    Bv    Bv

r 1  va r
 a r  v  br   
  br 
    
   2 2 2 2  
N s 1 j 1
  Qmi , j  2  Qmi , j  2  Qmi , j  2  Qmi , j  2  
B2
      B  B 
 zi1  z  zi  t  ti1  t
B   Bt
 i  
Qm    z
i 1
  
j 1   j   i 1   i  j
 
 n 1  4  Qmi , j Qmi , j 
 
 
 0 .15
va rj var ( j 1)
B B
v a rj v a r ( j 1)
 
 j 1  r 1   j  i

  Q  2 2
4   Q 
2 2
 2 
  eti  B 2   Qeti  B 2    eti  B 2   Qeti  B 2    Qeti  B 2 
n

  p  p  r  r  
j 1 t 1  va t 
 vat  v  vbt      
      bt  j   
 2 2 2 2 2

N s 1
  Qeti  2  Qeti  2  Qeti  2  Qeti  2  Qeti  2 
BQ2et       BDa    BD  
  BDo    BD  
  BDb 
i 1   Da 
 D   D 
 Do   Db 
p avg
 p   avg  
 2 2 
   Qeti  B 2   Qeti  B 2    0.15 Qeti Qeti B B 
n 1 4
 
  t  ti 1  t  ti   
  va rj var ( j 1)
v a rj v a r ( j 1 )
 
  i1   i  j 1 r 1 
  j 
 i
2 2 2 2 2
 Q   Q  2  Qel  2   2
B 2
  el  BV2   el  BL    BD   Qel  BD
 Q 
  el  B2
Qet  D   D
 Vel   Le   
el e avg ADCP
  avg   ADCP 
2 2 2 2
 2  Qer  2 2
 Q   Q  Qer  2  Q 
B 2
  er  BV2   er  BL    BD    BD   er  B2
Qer
 Davg    
 Ver   Ler  D ADCP
er er avg r ADCP
  r  
ADCP UA: Implementation

Uncertainty Propagation to Final Result: Precision Limit

tS Qt
PQt 
M

Uncertainty Propagation to Final Result: Total Uncertainty

U Qt  BQ2t  PQ2t
 ADCP UA: Implementation

Practical approach
(pitch and roll neglected in DRE; errors accounted through end-to-end calibrations)

 velocity (instrument coordinates neglecting the pitch and roll angle) 1

 (V  V ) 2  (V  V ) 2 (V  V ) 2
 (V  V ) 2 2
2 3 4 1 2
V f i 1 , j   3 4 2
1 2
 Vb  2   Vb  cos(   )
 4 sin  2 sin  

q = beam angle,
where
a = angle of the flow to instrument
β = angle of the boat velocity
V1 ~ V4 = in beam water velocities
Vb = boat velocity

 total discharge

n
N s 1

Qt    V f  i 1, j  Vb , i 1 sin( i 1, j 
  i 1 )  z j 1  z j  t i 1  t i    Qel  Qer
i 1  j 1 
 ADCP UA Software - architecture
• Developing tools - Borland C++ Builder (v.6) & Microsoft Access
• Software Configuration
ADCP Uncertainty Analysis & GUI Flow Chart

New Measurements Prior Information ( u1 ~ u19 )

Processing Database
ADCP ADCP Configuration File - Calibrations
- Vertical, Horizontal Velocity Profile
Output Output - operational and - Discharge (WinRiver & WinADCP - Manufacturer specifications
Binary ASCII environmental - Literature compilation
Homologuous)
Files Files specifications - Visualizations - prior measurements
- User UA archive

Statistical Analysis for UA

Error Propagation to Final Results

(Embedded in Uncertainty Analysis GUI)
- Velocity Uncertainty Analysis
- Discharge Uncertain Analysis

Uncertainty Analysis Graphical User Interfaces

Numerical Display
Graphical Display
Uncertainty Analysis Output

5
4
3
2
1
 ADCP UA Software - GUIs
Archive database
- Elemental uncertainties are archived in categories based on river characteristics.
- Users with limited level of preparedness can estimate uncertainties
using default values obtained in similar environment and operating conditions.
- The stored information is updated as soon as new measurements are processed.
- User can also create new archives using new classification categories
ADCP UA Software - GUIs
Information for archiving

Discharge

Channel
Profile

Vessel Moving Path

Flow Direction
River
Characteristics
 ADCP UA Software - GUIs
Assessment of bias limit
Individual Error Source Input
Default Values based
on Archive Database

River Characteristics

Relevant Literature
for error sources
 ADCP UA Software - GUIs
Assessment of precision limit
Variation of Discharge
Measurements

Repeated Measurements
for a transect

Result
 ADCP UA Software & GUIs
Assessment of total uncertainty

Calculated Uncertainty in
Discharge Measurement
 Conclusions

 Feasibility of UA engineering standards for implementation to

ADCP measurements
 The methodology is comprehensive, simple to implement
 Easily upgradeable as new info occur
 UA allows tracing of the measurement accuracy to primary
standards  withstand legal and strict QA/QC requirements
 Finalization of UA – an extensive and expensive effort
Collaboration between manufacturers and users in a coordinated
effort = key to complete UA for the variety of measurement
situations and operating conditions encountered in monitoring
practice
 The framework was adopted by ASCE’s HME Task Committee and
the UNESCO group on Data Requirements for Integrated Urban
Water Management (Fletcher et al., 2007)
 Currently evaluated by the ISO committee (Herschy)
 Conclusions

 The UA customized software for ADCP velocity and

discharge measurements requires minimum user
preparation

 Autoarchiving uncertainties for specific environments and

operating conditions can provide information about
dominant sources of uncertainties at various sites.

 By continuously increasing the sample size through

archiving, the UA output is progressively enhanced.
 Outlook

 Work closely with manufacturers and users to assess

elemental error sources (manufacturer, operator, environment,
or combinations) and integrate them in the AIAA (1995)
uncertainty assessment framework for rigorous documenting
velocity and discharge measurement accuracy

 Conduct sensitivity analysis and field tests for compiling

uncertainty minimizations guidelines

 Develop operational guidelines for conducting accurate

measurements in various flow regimes
 Outlook

 Need for coordination and extensive collaboration among

ADCP manufacturer, operators, data users, and third-party
evaluators

 Need for evaluation of the status of current developments

and to strategize for integrative efforts to assess
methodologies for operation and accuracy assessment of
the ADCP as well as other flow measurement techniques
over an extend the range of flow conditions (present WMO
effort)

 IIHR is willing to be actively involved in the WMO initiative

Thank you!

Questions?