Professional Documents
Culture Documents
Data Reconciliation
Data Reconciliation
Module:
Introduction to Data Reconciliation
Tier 1:
Basic Concepts in Data
Reconciliation
Chapter 1:
General Introduction
y = x + e (1.1)
Two Components:
Random Error,
Gross Error, es
y = x + es +
Gaussian
distribution
4
Gross error
0
-4 M easurem ents
T rue
-8
0 20 40 60 80 1 00
Tim e
Estimator or Filter
Measurements Estimates
For example,
CWS: Cooling FI FI FI FI
Water Supply CWS CWR
1 3 5 6
CWR: Cooling FI FI
Water Return
2 4
Plant 2
The estimation of the true values for the flows in this network
can be solved by Date Reconciliation (DR).
Process models
Monitoring Management
Optimization Modeling
Simulation Control
Instrument maintenance Equipment analysis
Optimization
Refinery Petrochemical
Metal/Mineral Chemical
Pulp/Paper
minimizing J ( yˆ , zˆ ) ( y yˆ ) V ( y yˆ )
T 1
(1.3)
subject to f ( yˆ , zˆ ) 0
g ( yˆ , zˆ ) 0
Nonlinear
Linear algebraic Nonlinear
Steady-State DR
models algebraic
models
Linear
Steady-State DR
Data
Reconciliation
Techniques
Unmeasured variables
Nonobservable
Figure 1.7: Classification of process variables
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
1.4 Process Variable Classification
CHAPTER 1
A redundant variable is a measured variable that can be
General estimated by other measured variables via process models, in
Introduction addition to its measurement.
In Figure 1.4, all six flows are measured, and any one of them
can be estimated by mass balances using other measured
flows, so they are all redundant variables.
General
Introduction Plant 1 Plant 3 Plant 4
FI FI FI
CWS CWR
1 3 5 6
2 4
Plant 2
2, 4, and 6 eliminated.
Question 2:
The effects of a systematic measurement bias on the
estimation of a process
(a) is more significant than that of random error.
(b) can be negligible compared with that of random error.
(c) is compatible to a random error.
(d) can be eliminated provided that it is detected.
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
1.6 Quiz
CHAPTER 1
Question 3:
General
Introduction Data reconciliation uses information from
(a) process models.
(b) process measurements.
(c) human common sense.
(d) redundancy in measurements.
Question 4:
If a process variable is measured, then
(a) it is observable.
(b) it is unobservable.
(c) it is maybe redundant.
(d) it is maybe nonredundant.
http://www.ris-resolution.com/reconciliation.shtml
http://www.thp.be/reconciliation_nl.html
http://btbjansky.com/prozessdat_e.html
http://www.simsci.com/products/datacon.stm
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
Chapter 2:
Linear Steady-State Data
Reconciliation with All Variables
Measured
F 60.8
Plant 1 Plant 3 Plant 4
2
F 35.0
FI FI FI FI y
3
.
CWS CWR F 68.9
4
1 3 5 6 F 38.6
5
FI FI
F 101.4
6
2 Plant 2 4
Figure 1.4
and 0 is a zero-vector.
1 -1 -1 0 0 0 1
0 1 0 -1 0 0 2
A
0 0 1 0 1 0 3
0 0 0 1 1 1 4
Incidence matrix: A
Variables
Measured subject to Ayˆ 0.
Ay AVA T 0 (2.5)
E( yˆ ) E( x ε ) E[ VA ( AVA ) A( x ε )]
T T 1
(2.10)
Reconciliation
with All Since E() = 0, and x is a deterministic variable, E(x) = x, thus:
Variables
E( yˆ ) x VA ( AVA ) Ax
T T 1
Measured
And since Ax = 0 (the true values of the flows satisfy mass
balances), VAT(AVAT)-1Ax = 0. Therefore:
E( yˆ ) x (2.11)
Question 2:
The reconciled data in linear steady-state DR are
(a) more consistent, but less accurate than raw
measurements.
(b) more accurate, but less consistent than raw
measurements.
(c) more accurate and consistent than raw measurements.
(d) less accurate and consistent than raw measurements.
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
2.3 Quiz
CHAPTER 2
Question 3:
Linear Steady-
State Data The reconciled data in linear steady-state DR
Reconciliation (a) are biased.
with All (b) are unbiased.
Variables (c) have a smaller variance than the raw measurements.
Measured
(d) have a larger variance than the raw measurements.
Question 4:
The incidence matrix of a flow sheet
(a) is unique.
(b) is not unique.
(c) contains topological information about a flow sheet.
(d) contains the measurement information of a flow sheet.
Chapter 3:
Linear Steady-State Data
Reconciliation with Both Measured
and Unmeasured Variables
A yˆ A zˆ 0
y z
(3.1)
A yˆ Fˆ
3
0 1 1
y
F̂
5
0 0 1
FI Plant 1 FI Plant 3
FI Plant 4
CWS CWR 1 0 0
F̂
3 5 6 1 1
1 0
2
Plant 2
A zˆ Fˆ
4
0 0 0
z
F̂
2 4 6
Measured and
Unmeasured
The solution to the data reconciliation problem (3.2) can be
Variables
solved by first eliminating the unmeasured flows, ẑ , in the
constraint equations by pre-multiplying both sides by a
projection matrix , P, such that PA 0 . Then the data z
minimizing J ( yˆ ) ( y yˆ ) V ( y yˆ )
T 1
(3.3)
subject to PA yˆ 0
y
Measured and
Unmeasured The construction of the projection matrix, P, can be performed
Variables efficiently using Q-R factorization of matrix Az.
Linear Steady-
State Data
Reconciliation R1 is an upper triangular and nonsingular matrix with
with Both dimension (nn). 0 is a zero matrix with the dimension
Measured and (m-nn). I is an identity matrix.
Unmeasured
Variables After the Q-R factorization of matrix A , the matrix Q can be
z
partitioned into two parts as:
R
A Q Q 1
0
z 1 2
(3.5)
Reconciliation 0
2 2 1
(3.6)
2
with Both
Measured and Since Q is orthonormal, the matrix Q has the property:
2
Unmeasured
Variables R R
Q Q Q 0 I 0
T 1 1
0 0
2 1 2
Thus Q2TA = 0.
P = Q 2T
Q 1
Q 2
P = Q2T = [0 0 1 0]
and we have: 1 1 0
0 0 0
PA 0 0 1 0 0 1 1
0 1 1
y
0 0 1
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
3.1 Solutions to Estimates of Measured Variables
CHAPTER 3
Note that the first element in the matrix PAy is zero. This
Linear Steady-
indicates that the measurement F1 will disappear in the mass
State Data
Reconciliation balance of the constraint equations in (3.3). The measurement
with Both F1 is nonredundant. This means it can only be evaluated by its
Measured and measurement.
Unmeasured
Variables Now, the data reconciliation becomes to reconcile the two
redundant measurements, F3 and F5. Rewrite the problem as:
J ( yˆ ) ( y yˆ ) V ( y yˆ )
T 1
minimizing
Ayˆ 0
subject to
F 35.0 0.2116 0
y 3
V A 1 -1
where F 38.6 ,
5 0 0.2025 , .
Measured and Note that the reconciled values satisfy the mass balance at
Unmeasured plant 3. The estimates for the three measured flows by the DR
Variables algorithm are:
F̂ 110.5
1
yˆ Fˆ 36.84
3
F̂ 36.84
5
The quantities on the right side of (3.7) are known, so now the
problem is to solve the linear equations on the left side.
Usually, the number of equations is greater than the number of
unmeasured flows. The least-squares technique can then be
applied and give the solution of the observable unmeasured
flows as:
FI FI FI
CWS CWR
1 3 5 6
2 4
Plant 2
Measured and
Unmeasured F 2
F
Variables F
where y , and z . The two partitioned matrices
1 3
are: F 6
F 4
F 5
1 0 1 1 0 0
0 0 1 0 1 0
A A
0 0 (42) , 0 1 0 1 (44) .
y z
0 1 0 0 1 1
0 0
1 2
with Both 2
Measured and
Unmeasured
Then we have: 1 0
Variables 0 0 ˆ
PA yˆ 0.5 0.5 0.5 0.5 F Fˆ Fˆ 0
1
0 0 Fˆ
y 1 6
6
0 1
z
N-r
Q
T
Since Q Q
T
T
Q , the term QTAy in (3.10)
1
Q
1 2 T
2
Q A
T
Q A T 1 y
can be written as: Q A y T
2 y
Reconciliation Q A Q Q Q 0 0 Q Q Q Q 0 0
T 1 1 2 1 1 1 2 1 2
z T 1 2 T T
2 2 1 2 2
with Both
Measured and Because Q is an orthonormal matrix, Q Q I , Q Q 0 , T T
1 1 1 2
Unmeasured Q Q 0 , and Q Q I . Therefore,
T T
Variables 2 1 2 2
I 0 R R R R
Q A
T
1 2 1 2
0 I 0 0 0 0
z
0 (3.11)
zˆ
1 y 1 2
Q A
0 0
T r
2 y
zˆ
Nr
Reconciliation
with Both Q A yˆ 0 T
2 y (3.13)
Measured and Note that equation (3.13) is the reduced form of the mass
Unmeasured balances by the projection matrix, Q . T
2
Variables
zˆ R Q A yˆ R R zˆ
r
1
1
T
1 y
-1
1 2 N-r (3.14)
unknown (nonobservable).
Reconciliation 1 2
zeroes.
with Both
Measured and
Unmeasured For the example of the cooling water network shown in
Variables Figure 3.2, the vector of the unmeasured flows is decomposed
as:
F 2
z F z r
z r 3
z F
N-r 4
F z
5 N-r
RR
-1
1 2
0 0
z 1 2
Measured and
Unmeasured where is a permutation matrix having the property
Variables
T = = -1 . For this example, the permutation is:
1 0 0 0
1 1 0 0 1 0 1 0
0 0 1 0
A 1 1 0 0 1 0 1 0 .
z
0 1 0 0
0 0 1 1 0 1 0 1
0 0 0 1
A yˆ A zˆ 0
y z
T
(Note: T=I)
A yˆ ( A )( zˆ ) 0
y z
T
zˆ
Measured and
N-r
Unmeasured where the variables in the subset ẑ are the minimum number
N-r
I am redundant.
I am redundant,
too.
FI FI FI
CWS CWR
1 3 5 6
I am
nonredundant. 2 Figure 3.1
4
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
3.3 Observabilty and Redundancy Analysis
CHAPTER 3
The problem of linear steady-state data reconciliation with both
Linear Steady- measured and unmeasured variables can be efficiently solved
State Data using the projection matrix method. This technique is
Reconciliation summarized in the following steps.
with Both
Measured and
Unmeasured Step 1: Decompose the system matrix, A, in terms of Ay and
Variables Az, which correspond to measured and unmeasured variables.
Unmeasured
Variables
Step 5: Get the projection matrix, P = Q2T. Proceed with the
data reconciliation using Equation (3.3). Only redundant
measured variables participate in the data reconciliation. The
nonredundant measurements are identified by the matrix Q2TAy
. Obtain the estimates for the measured variables.
Question 2:
If the rank of Az is r < N, where N is the total number of
unmeasured variables, then
(a) at least (N-r) variables are nonobservable.
(b) there are exactly (N-r) variables nonobservable.
(c) there are r variables observable.
(d) there are r variables nonobservable.
© North Carolina State University, USA © University of Ottawa, Canada, 2003
Basic Concepts in Data Reconciliation
3.4 Quiz
CHAPTER 3
Question 3:
Linear
Steady-State Comparing the two Equations (3.8) and (3.14) ,
Data (a) they are equivalent.
Reconciliatio (b) equation (3.8) is used only when all variables are observable.
n with Both (c) equation (3.14) can always be applied whether all variables
Measured and are observable or not.
Unmeasured
(d) equation (3.14) can never be applied whether all variables are
Variables
observable or not.
Question 4:
For the case where the unmeasured variables can be calculated
by Equation (3.8), show that the expected values of the estimates
are
E(zˆ ) ( A A ) A A x
T
z z
1 T
z y y