Formulas and statistical tables for CHEM2241

Buoyancy effect of air on an ρ air

Bobject = M object ×
object ρ object

 ρ air ρ air 
True weight of water M corrected = M w ×  1 + − 
 ρ water ρ weight 

suspect value − nearest value
Q-test Qobs =
largest value − smallest value

Confidence limit (single µ = x ± zσ

measurement)

Confidence limit (N zσ ts
µ=x± , µ =x± where tN-1
measurements) N N

x1 − x2 N1 N 2 x−µ
t-test t= , t= N
s pooled N1 + N 2 s

( N1 − 1)( s12 ) + ( N 2 − 1)( s 22 )

Pooled standard deviation s pooled = ; where tN1+N2-2
( N1 − 1) + ( N 2 − 1)

F-test F = s12/s22

SS resid SS regr
R2 = 1− = , SS tot = ∑ ( yi − y ) 2 ,
Coefficient of determination R2 SS tot SS tot
SS resid = ∑ [ yi − (b + mxi )]2

Standard deviation for results 𝑆𝑆𝑐𝑐 =

𝑆𝑆𝑟𝑟 1
� + +
1 (𝑦𝑦�𝑐𝑐 − 𝑦𝑦�)2 𝑆𝑆 2
; 𝑆𝑆𝑥𝑥𝑥𝑥 = � 𝑟𝑟 � ; where tN-2
𝑚𝑚2 𝑆𝑆𝑥𝑥𝑥𝑥
from calibration curve, Sc: 𝑚𝑚 𝑀𝑀 𝑛𝑛 𝑆𝑆𝑚𝑚

3σ 3σ
Upper and lower control limits UCL = µ + , LCL = µ −
N N

Indeterminate errors in chemical 2 2 2 2

= s 2population + s sampling
2
soverall = s sample + s measurement, s sample
analysis

Size of gross sample for a two-

component system

[𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏] [𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎]

Henderson-Hasselbalch equation: 𝑝𝑝𝑝𝑝 = 𝑝𝑝𝐾𝐾𝑎𝑎 + log 𝑝𝑝𝑝𝑝𝑝𝑝 = 𝑝𝑝𝐾𝐾𝑏𝑏 + log
[𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎] [𝑏𝑏𝑏𝑏𝑏𝑏𝑏𝑏]

2 2 2
Number of samples for  ts   ts   ts 
acceptable level of uncertainty u
N =  o  =  o  =  o 
 µ − x   u   xσ u 
𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑜𝑜𝑜𝑜 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎
Gravimetric factor Gravimetric factor =
𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑜𝑜𝑜𝑜 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝

Page 1 of 4
t Values for Calculating Confidence Limits of Averages

No. of Degrees of t for 90% t for 95% t for 99%

measurements, n freedom, n – 1 probability probability probability
2 1 6.3 12.7 63.7
3 2 2.9 4.3 9.9
4 3 2.35 3.2 5.8
5 4 2.13 2.78 4.6
6 5 2.02 2.57 4.03
7 6 1.94 2.45 3.71
8 7 1.90 2.37 3.50
9 8 1.86 2.31 3.36
10 9 1.83 2.26 3.25
11 10 1.81 2.23 3.17
12 11 1.80 2.20 3.11
13 12 1.78 2.18 3.06
14 13 1.77 2.16 3.01
15 14 1.76 2.15 2.98
16 15 7.75 2.13 2.95
21 20 1.72 2.09 2.84
31 30 1.70 2.04 2.75
∞ ∞ 1.64 1.96 2.58

Values of F at the 95% Confidence Level

Degree of Degree of freedom (numerator)

freedom
(denominator) 2 3 4 5 6 12 20 ∞

2 19.00 19.16 19.25 19.30 19.33 19.41 19.45 19.50

3 9.55 9.28 9.12 9.01 8.94 8.74 8.66 8.53
4 6.94 6.59 6.39 6.26 6.16 5.91 5.80 5.63
5 5.79 5.41 5.19 5.05 4.95 4.68 4.56 4.36
6 5.14 4.76 4.53 4.39 4.28 4.00 3.87 3.67
12 3.89 3.49 3.26 3.11 3.00 2.69 2.54 2.30
20 3.49 3.10 2.87 2.71 2.60 2.28 2.12 1.84
∞ 3.00 2.60 2.37 2.21 2.10 1.75 1.57 1.00

Q-Test Values for Three to Ten Measurements at the 95% Confidence Level

n 4 5 6 7 8 9 10
Q95% 0.76 0.64 0.56 0.51 0.47 0.44 0.41

Page 2 of 4
Error Propagation in Arithmetic Calculations
Type of Example Standard Deviation of y
Calculation
Addition or y = a+b−c
Subtraction
s y = sa2 + sb2 + sc2
2 2 2
Multiplication sy s  s  s 
y = a ⋅b / c =  a  + b  + c 
or Division y a b c
sy sa
Exponentiation y = ax =x
y a
sa
Logarithm y = log10 a s y = 0.434
a
sy
Antilogarithm y = antilog10 a = 2.303sa
y

Page 3 of 4
Page 4 of 4