Statistical Measurement in

Chemical and Bioprocess Analysis

JCB40104 – CHEMICAL AND BIOPROCESS INSTRUMENTATION

Significant Figure
3
4
5
6
7
 Significant Figures Practices
How many significant figures in the following?

1.0070 m  5 sig figs

17.10 kg  4 sig figs

010,890 L  5 sig figs

3.29 x 103 s  3 sig figs

0.0054 cm  2 sig figs

3,200,000 ml  7 sig figs

6.990 x 107 mA  4 sig figs 8

Statistical Measurement in Chemical and
Bioprocess Analysis
Once a measurement has been repeated, a statistical exploitation becomes
possible. However, the laws of sampling and tests based upon hypotheses must be
understood to avoid non-value conclusions, or to ensure the meaningful quality
tests.

Systematic errors (user-based, instrumental) or gross errors which lead to results

beyond reasonable limits do not enter into this chapter. For the tests most frequently
met in chemistry only indeterminate errors are considered here.
 3 Main Statistical Measurement
in Chemical and Bioprocess
Instrumentation

REJECTION
ACCURACY OF DATA
(OUTLIER)

PRECISION

Mean Q-Test /
% Error Dixon Test

Standard
Deviation
Mean Value, Accuracy and
Error
MEAN: average
ACCURACY: refers to the closeness of a measured value to a standard or
known value. 
Accuracy is how close a measurement comes to the truth, represented
as a bullseye above.  Accuracy is determined by how close a
measurement comes to an existing value that has been measured.
Accuracy is the ability of an instrument to show the exact reading. To
determine if a value is accurate compare it to the accepted value. 
Accuracy normally shown in percentage of error which of the full scale
reading percentage.

% error =
exercise
You are given a cube of pure copper. You measure the sides of
the cube to find the volume and weigh it to find its mass. When
you calculate the density using your measurements, you get 8.78
grams/cm3. Copper’s accepted density is 8.96 g/cm3. What is your
percent error? Solution:
experimental value = 8.78 g/cm3
Accepted Value accepted value = 8.96 g/cm3
- the correct Step 1: Subtract the accepted value from the experimental value.
value based on 8.96 g/cm3 – 8.78 g/cm3 = -0.18 g/cm3
reliable
Step 2: Take the absolute value of step 1
references
|-0.18 g/cm3| = 0.18 g/cm3

Experimental Step 3: Divide that answer by the accepted value.

Value
- the value                                   
measured Step 4: Multiply that answer by 100 and add the % symbol to
express the answer as a percentage.
0.02 x 100 = 2%
Precision & Standard
Deviation
Precision is how close a measurement comes to another measurement.
Precision is determined by a statistical method called a standard
deviation.  The standard deviation is called ‘a dispersion index’.
Standard deviation is how much, on average, measurements differ from
each other. The mean deviation d or mean of the differences,
calculated over n measurements allows an appreciation of the
precision.

High standard deviations  low precision,

low standard deviations  high precision.
To illustrate accuracy and precision the standard deviations have been calculated.
Note the differences between the arithmetic means indicated on the graph by a
short vertical bar and the corresponding median values, which are arrowed.
For the results from chemists 3 and 4, the difference is fairly large. Chemist 1 has
committed very probably a systematic error. On the right, a classic illustration of the
precision and accuracy depicted with the aid of a target. This image is less simple
than it would appear because there remains an uncertainty in both x and y.
 EVALUATE THE ACCURACY AND PRECISION

ACCURATE NOT ACCURATE

& &
PRECISE PRECISE

ACCURATE NOT ACCURATE

& &
NOT PRECISE NOT PRECISE
Rejection of possible Outlier
Data (Q- / Dixon test)
Sometimes, a value within a set might appear abnormal. Although it
might be tempting to reject this data point, it must be remembered that
it is only abnormal in respect of a given law of probability. There exists a
simple statistical criterion for conservation or rejection of this outlier
value.
This is Dixon’s test, which consists of calculating the following ratio.
If Qcalc is greater than Qtable the value in question can be rejected

If Q >Q :
calc table Data rejected
 Example:: Is 167 an outlier in this set of data? Test at the 95% confidence
Level..

167, 180, 188, 177, 181, 185, 189

 Step 1: Sort your data into ascending order (smallest to largest).
167, 177, 180, 181, 185, 188, 189
 Step 2 : Find the Qcalc using the formula
 Step 3 : Find the Qtable value in the Q table. For a sample size of 7 and an
alpha level of 5%, the critical value is 0.568
 Step 4 : Compare the Qcalc from Step 2 with the Qtable value in Step 3. If the
Qcalc is greater than the Qtable value, the point is an outlier.
Qcalc = 0.455
Qtable = 0.568
 Solution: 0.455 is not greater than 0.568, so this point is not an outlier at
confidence level of 95% (accept).
TRY OUT..

Example 1:
Data obtained for the measurement of Hg2+ ions in waste water sample
are 5.3 ppm, 5.0 ppm, 4.7 ppm, 4.4 ppm, 5.5 ppm and 3.2 ppm.
Identify the possible outlier and evaluate its rejection criteria at 95%
confidence level.
 QUIZ TIME..

0.189, 0.167, 0.187, 0.183, 0.186, 0.182,

0.181, 0.184, 0.181, 0.177

From the data series above,

1. Identify the questionable data
2. Evaluate the data by using Q test at 90% and 95% confidence
level.
3. From the obtained result in (2), Calculate their respective
mean and standard deviation.
 !
CA
P 3 Main Statistical Measurement in
RE Chemical and Bioprocess
Instrumentation

REJECTION
ACCURACY OF DATA
(OUTLIER)

PRECISION

Mean Q-Test /
% Error Dixon Test

Standard
Deviation
What is the relationship of
Significant Figure to the
Accuracy and Precision?