Static Calibration Process

Ref: ANSI / ISA (1979) standard

Chapter 2
 Key Steps
• Step 1: Apply known values of the measurand to the measuring system.

• Step 2: The measurand values are incrementally increased from the low
end of the range to the top end

• Step 3: The measurand values are then incrementally decreased to the

low end of the range

• Step 4: Calculate key parameters of system such as accuracy, etc.

The device is allowed to come to equilibrium prior to taking a reading

Example: Calibrate this spring scale
• Range is 0 to 5 lb
 Example
        scale reading    
True weight cycle 1 cycle 2 cycle 3 cycle 4 cycle 5 cycle 6
0.5 0.2 0.08 0.17 0.19 0.11
1 0.7 0.78 0.64 0.61 0.7
1.5 1.18 1.26 1.25 1.24 1.23
2 1.81 1.93 1.81 1.93 1.88
2.5 2.62 2.49 2.46 2.46 2.58 2.53
3 3.15 3.18 3.24 3.28 3.13  
3.5 3.9 3.84 3.86 3.97 3.96  
4 4.59 4.71 4.61 4.6 4.6  
4.5 5.41 5.35 5.49 5.46 5.39  
5 6.24 6.27 6.1 6.24 6.16  
4.5 5.71 5.74 5.78 5.87 5.82  
4 4.96 5.11 5.08 5.03 5.03  
3.5 4.22 4.34 4.21 4.22 4.24  
3 3.57 3.64 3.66 3.55 3.67  
2.5 2.98 2.86 2.98 2.98 2.94  
2 2.22 2.23 2.26 2.29 2.26  
1.5 1.57 1.7 1.69 1.63 1.57  
1 1.07 1.07 1.11 1.16 1.11  
0.5 0.52 0.61 0.61 0.61 0.45  
0   0.02 0.08 0.08 -0.03 0.06  
 Curve Fit Data
• First, we try to fit a linear model through the
data.
1 n
x  sample mean of the x values   xk
n k 0
a1 
 xy    x   y 
1 n
y  sample mean of the y values   yk  x   x
2 2

n k 0

1 n
a0 
 
x2 y    x   xy 
x 2  sample mean-square of the x values  
n k 1
xk2
 x   x
2 2

y  a1 x  a0
1 n
xy  sample mean of the product xy   xk yk
n k 1
 Curve Fit Data
• First, we try to fit a linear model through the
data.

a1 
 xy    x   y 
• Should we use data from one
cycle or all cycles ?
 x   x
2 2

a0 
 
x2 y    x   xy 
 x   x
2 2

y  a1 x  a0
 Curve Fit Data
7

4
Axis Title

Linear ()

0
0 1 2 3 4 5 6

-1

Axis Title
 Curve Fit Data
• We get:
Reading (R)= (1.29) . True Weight (W) – 0.37

Now we can predict values of readings for every experiment

Output Span: Difference of max of predicted values and min

of predicted values

We shall later define accuracy in terms (as %age) of output

span
 Curve Fit Data
• We get:
Reading (R)= (1.29) . True Weight (W) – 0.37

Let’s see deviation of our predicted values to

true values
Deviation
True wgt cycle 1 cycle 2 cycle 3 cycle 4 cycle 5 cycle 6
Table 0  
0.5 -0.07 -0.19 -0.10 -0.08 -0.16
1 -0.22 -0.14 -0.28 -0.31 -0.22
1.5 -0.38 -0.30 -0.31 -0.32 -0.33
2 -0.40 -0.28 -0.40 -0.28 -0.33
2.5 -0.23 -0.36 -0.39 -0.39 -0.27 -0.32
3 -0.35 -0.32 -0.26 -0.22 -0.37 
3.5 -0.24 -0.30 -0.28 -0.17 -0.18 
4 -0.20 -0.08 -0.18 -0.19 -0.19 
4.5 -0.02 -0.08 0.06 0.03 -0.04 
5 0.16 0.19 0.02 0.16 0.08 
4.5 0.28 0.31 0.35 0.44 0.39 
4 0.17 0.32 0.29 0.24 0.24 
3.5 0.08 0.20 0.07 0.08 0.10 
3 0.07 0.14 0.16 0.05 0.17 
2.5 0.13 0.01 0.13 0.13 0.09 
2 0.01 0.02 0.05 0.08 0.05 
1.5 0.01 0.14 0.13 0.07 0.01 
1 0.15 0.15 0.19 0.24 0.19 
0.5 0.25 0.34 0.34 0.34 0.18 
0 0.39 0.45 0.45 0.34 0.43 
 Analysis
Output Span is 6.076 - (-0.374) = 6.45 lb
– Put W=0 and W=5 in R=1.29W-0.37 equation

Accuracy limits are 0.45 lb and -0.40 lb i.e. max/min of dev plot
so it is 7% and -6.2 % of output span

Terminal Linearity: if we draw a straight line connecting extreme points of avg

up-dn curve, linearity error is max deviation b/w line and avg up-dn curve.

Hysteresis error:
For every cycle and for each weight, find the difference b/w up and
corresponding dn reading. Hysteresis is the max of all these values.

Repeatability : maximum variability of successive measurements of same input

approached from same direction
0.6
Dev v. True Weight
0.4

0.2

0
0 1 2 3 4 5 6
0.50

0.40

0.30

0.20

0.10 avg of up-dn

avg of dn
0.00 avg of up
0 1 2 3 4 5 6

-0.10

-0.20

-0.30

-0.40
 Example
• A temp measuring device has a range of 5 to
25oC. Calibration data for 3 cycles is given
below:
 
 Sensor Reading  
Env Temp cycle 1 cycle 2 cycle 3
Find:
5 1 2 1
i) Output span
15 13 12 13
ii) Accuracy Limits
iii) Hysteresis 25 26 25 24
iv) Terminal Linearity
v) Repeatability 15 16 17 17
5   5 4 6
Temp Reading 1 n
x  sample mean of the x values   xk
5 1 n k 0
15 13
1 n
25 26 y  sample mean of the y values   yk Y-bar
15 16 n k 0
12.13333
5 5 xy-bar
n
1
5 2
x 2  sample mean-square of the x values  
n k 1
xk2 219.33
x-bar
15 12 13
1 n
xy  sample mean of the product xy   xk yk x2-bar
25 25 n k 1
225
15 17
a1 
 xy    x   y  a1

 
5 4 1.100
 
2
x 2
 x a0
5 1 -2.167
15 13
25 24 a0 
 x   y    x   xy 
2

15 17  x   x
2 2

5 6 y  a1 x  a0
 Hysteresis = 5 or
R = 1.1 T - 2.167 22.72% of o/p span

R ( 5) = 3.33, R (25) = 25.33 Accuracy Limits = [-2.33 2.67]

Output Span = 22 = -10.59 % and 12.14% of o/p span
Deviation Table Hysteresis
Env Temp cycle 1 cycle 2 cycle 3 Repeat Env Temp cycle 1 cycle 2 cycle 3
5 -2.33 -1.33 -2.33 1.00 5 -4 -2 -5
15 -1.33 -2.33 -1.33 1.00 15 -3 -5 -4
25 0.67 -0.33 -1.33 2.00 25
15 1.67 2.67 2.67 1.00
5   1.67 0.67 2.67 2.00 For repeatability, max variation
happens at 5 (dn) i.e. 2 which is
2 x 100 / 22 = 9% of o/p span
 Deviation Table
Env Temp cycle 1cycle 2cycle 3
5 -2.33 -1.33 -2.33
15 -1.33 -2.33 -1.33
25 0.67 -0.33 -1.33
15 1.67 2.67 2.67
5   1.67 0.67 2.67

5 Cycle 1
Cycle 2
Cycle 3
0 5 10 15 20 25 30

2 4

Hysteresis
-
Env Temp cycle 1 cycle 2 cycle 3
5 -4 -2 -5
15 -3 -5 -4
25
-
 R = 1.1 T - 2.167
Hysteresis = 5
R ( 5) = 3.33, R (25) = 25.33
Accuracy Limits = [-2.33 2.67]
Output Span = 22
Deviation Table
Env Temp cycle 1 cycle 2 cycle 3 Repeat avg-cycle avg-up-dn avg-dn avg-up
5 -2.33 -1.33 -2.33 1.00 -2.00 -0.17 1.67 -2.00
15 -1.33 -2.33 -1.33 1.00 -1.67 0.33 2.33 -1.67
25 0.67 -0.33 -1.33 2.00 -0.33 -0.33 -0.33 -0.33
15 1.67 2.67 2.67 1.00 2.33
5   1.67 0.67 2.67 2.00 1.67

Hysteresis
Env Temp cycle 1 cycle 2 cycle 3
5 -4 -2 -5
15 -3 -5 -4
25
3.00

2.00

1.00

avg of up-dn
0.00 avg of dn
0 5 10 15 20 25 30 avg of up

-1.00

Terminal
Linearity
-2.00 =?
or
? % of
-3.00 o/p span