Professional Documents
Culture Documents
Statistical Guidelines For Collecting KPI Data
Statistical Guidelines For Collecting KPI Data
n
f f
f
n
f f
f
) 1 (
96 . 1 ,
) 1 (
96 . 1
with 95% probability.
In other words, if we want a confidence interval [f e, f + e] for p, the required
sample size is approximately
) 1 (
96 . 1
2
2
f f
e
n =
Example:
The Attach Failure Ratio for packet-switched is estimated at f = 10% with a sample
size of n
1
= 200 measurements. By the formula for p, the uncertainty of the
measurement at the 95% confidence level is e
1
= 4.16%. Thus, with 95% probability,
the true proportional frequency p falls within the interval [5.84%, 14.16%].
To reduce the uncertainty to 3% at the same confidence level, a sample size of n
2
= (1.96
2
/ 0.03
2
) 0.1 0.9 = 384 would be needed.
2(2) Public
In practice, at least 200 measurements should be performed for all KPIs defining
success or failure ratios, and preferably 500 if at all practicable.
2.2. Estimation of Time, Data Rate, and Other
Quantities
The remaining KPIs in TEMS Investigation constitute estimates of some quantity p
(e.g. setup time, data rate, quality) in the form of a measurement average p.
According to the central limit theorem, the distribution of the underlying
measurement samples is asymptotically Gaussian. Therefore the uncertainty of the
estimation (again assuming a 95% confidence interval) is
n s e = 96 . 1
where the standard deviation s is the square root of the variance
( )
1
2
2
=
n
x x
s
and n is the sample size. The sample size corresponding to a predefined estimation
accuracy (confidence interval [f e, f + e]) can therefore be calculated as
2
2 2
96 . 1
e
s
n =
If measurements show a higher standard deviation than expected, the sample size
must be increased further in order to give the desired accuracy.
Example:
Attach Setup Time is estimated from a sample of size n
1
= 200. The measured
standard deviation is s = 250 ms. Then e
1
= 1.96 250 / 200
1/2
= 49 ms. If the
measured average Attach Setup Time is 1820 ms, we can be 95% confident that the
true average Attach Setup Time is between 1771 and 1869 ms.
To shrink the confidence interval so that e
2
= 20 ms, a sample size of n
2
= 1.96
2
250
2
/ 20
2
= 600 would be required.