Advanced Diploma

Research Methodology and Project (RMP470S)

EXPERIMENTAL WORK

 Dr N. Dlangamandla
 Email: dlangamandlan@cput.ac.za
 04 October 2023
EXPERIMENTAL WORK

- ERRORS
- PRESENTATION OF RESULTS

Special thanks to:

Prof Haldenwang and Mr Sutherland
4.1 Introduction
• The challenge in research is to prove observations and conclusions are
beyond reasonable doubt.
• Engineering research is numerically based and numbers are a prime
outcome
• Some research is qualitatively based – one has to convert qualitative
results to numbers
• Regardless of the research methodology, researchers must pay
attention to the use of statistics and the measure of uncertainty in their
research methods and in development of conclusions
• You should have done statistics in previous years
• Engineers commonly over-specify requirements in order the ensure
reliability of their designs
• Research outcomes which do not anticipate probability of failure are
better described as scientific investigations rather than engineering
research.
•Primary objective of engineering is the
improvement of the human conditions
•All engineering disciplines have an ethical
obligation to ensure human safety
•This section investigates the concept of the
appropriate level of accuracy required for
research outcomes to be acceptable in
human society.
•Failure to engage in the simplest of
statistical and error analysis reflects very
badly on th equality of the research, the
researcher and the conclusions
4.2 Sources of error and uncertainty

• Some errors occur randomly and cannot be

controlled
• Most errors can be controlled
• Through calibration
• Prediction
• Before measurement strategy is implemented
• Known as Systematic Errors
• Most specifications of measurement systems relate
to a linear response
• Linearity of response must be quantified
 Terminology - 1
Error and uncertainty often used interchangeably – unfortunate,
but live with it
Important to distinguish between accuracy, precision and
resolution
Accuracy is a measure of how close a real (measured) value is in
agreement with the true or accepted value
Precision is an indication of the ability to reproduce a value with a given
accuracy from experiments repeated under the same conditions i.e. a
measure of the spread of results about the expected result (or mean
value)

Resolution is the smallest increment (change) that can be detected

Terminology - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 6

 Terminology - 2

Measurement error x is defined as the deviation of the

measured (real, untrue) output value xr from the ideal (true,
known) value xi at the input (to a measuring system) i.e.
x = xr – xi

• Important when executing an experiment to minimise this error

Terminology - 3…

12 March 2019 Experimental Work – Errors and presentation of results Slide 7

 Terminology - 3
Dotted line = accepted value

Precise & Imprecise

accurate & accurate

Imprecise &
Precise & inaccurate
inaccurate

(Hughes & Hase)

Terminology - 4…

12 March 2019 Experimental Work – Errors and presentation of results Slide 8

 Terminology - 4
Resolution is the smallest increment (change) that can be detected
Analogue instruments - resolution generally
corresponds to the minimum distance between two
ticks of the graduated scale
Analog stopwatch: tickmark = 1/10 s = resolution

Digital instruments - resolution corresponds to the unit

value of the least significant digit (LSD)
Digital stopwatch: two digits beyond the seconds =
1/100 sec= resolution

8 bit A to D with full scale of 10V has resolution of

10/28 = 0.0391V but with full scale of 2.5V resolution =
0.0098V – any input less than 0.0391V (0.0098V) will
not change indication

http://www.tutelman.com/golf/measure/precision.php
Terminology - 5…

12 March 2019 Experimental Work – Errors and presentation of results Slide 8

 Terminology - 5
Sensitivity of an instrument is the ratio of the output values
to the input values
- In other words it is a “measure” of how much the display
changes in response to the variable causing it to change

Example

Thermometer A has a
sensitivity 5 times higher
than thermometer B
(Fornasini, 2008 )

Precision - analogue…

12 March 2019 Experimental Work – Errors and presentation of results Slide 9

 Precision - analogue
Generally half a division, but not always
• Can overestimate error, so interpolate (estimate position) between
tick marks

(Hughes & Hase)

Precision - digital…

12 March 2019 Experimental Work – Errors and presentation of results Slide 10

 Precision - digital
Could be specified by manufacturer e.g. half of last digit
(equivalent to analogue)
• E.g. voltmeter 8.41V = 8.410 ± 0.005V
• Assumes appropriate rounding is done by instrument (rather than
truncation)
Conservative approach – use full last digit (LSD)
Digital stopwatch example: precision = 1/10 s in each case since
reaction time not better than this, so only repeatable to 1/10 s
eg pressure or mass
200.20, 200.30, 200.15, 200.10, 200.25 ≡ 200.20 (accuracy = ±0.20, precision = ±0.10,
resolution = 0.05, not 0.01)
200.20, 200.40, 200.10, 200.00, 200.30 ≡ 200.20 (accuracy = ±0.20, precision = ±0.20,
resolution = 0.10)

Errors…

12 March 2019 Experimental Work – Errors and presentation of results Slide 11

 Errors
Aim is to quantify and record errors associated with spread in
experimental measurements - can never be certain of exact
value
• Errors give characteristic range in which it is believed, with a specific
likelihood, that the exact value lies

Concentrate on critical variables – very little can be gained by

reducing “small” errors

Types of errors…

12 March 2019 Experimental Work – Errors and presentation of results Slide 12

 Types of errors
Systematic (bias)

Random (precision)

Mistakes/blunders

Systematic (bias) errors…

12 March 2019 Experimental Work – Errors and presentation of results Slide 14

 Systematic (bias) errors -1
Permanent errors which cause measurements to always be either
larger or smaller than the true value i.e. to differ in the same
direction from the true value

The smaller this difference the more accurate the measurement, so

accuracy of measurement is determined by systematic errors

Examples
• Zero, calibration, defective equipment, loading errors, limitations of
system resolution, consistently recurring human errors

Cannot be quantified statistically

Systematic (bias) errors…

12 March 2019 Experimental Work – Errors and presentation of results Slide 15

 Systematic (bias) errors - 2

(Furlong, 2010)

Random (Precision) errors…

12 March 2019 Experimental Work – Errors and presentation of results Slide 16

 Random (precision) errors
Repeated measurements scattered over a range around a mean
value - usually follow a certain statistical distribution. Can be caused
by e.g.
• Human, disturbances to equipment, fluctuating experimental conditions,
inadequate instrument resolution
Increasing the number of readings decreases the uncertainty
caused by these errors (averaging)
• The smaller the random error the more precise the measurement
Best estimate is the mean of all data points
• Error is associated with distribution of points (above and below) the mean
value (standard deviation)

Can be quantified statistically (std dev)

Mistakes/blunders…

12 March 2019 Experimental Work – Errors and presentation of results Slide 17

 Mistakes/blunders
Mistakes made during testing
• 2.34 instead of 2.43 (e.g. calibration factor)
• Misreading a scale
• Malfunction of equipment

Computational errors after testing

• Using wrong multiplier e.g. 10x rather than 100x
• Using the wrong units (metric/imperial)

Mean & Std deviation - 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 18

 Mean & standard deviation - 1
Best method of reducing effect of random errors on
measurement is to average

The mean is the average value of a set of readings, and gives the
best estimate of a value

It is given by

But this does not yield all information – need precision as well

(Hughes & Hase)

Mean & standard deviation - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 18

 Mean & standard deviation - 2
Maximum deviation can give misleading spread, so need a
measure that depends on all measurements

Deviation of measurement from mean can be positive or

negative and is given by
Average deviation 𝑑= 1 𝑁 𝑖 𝑑𝑖 = 1 𝑁 𝑖 𝑥𝑖 − 𝑥 = 𝑥- 1 𝑁 𝑁𝑥 = 0
• i.e. always equals zero, so it cannot indicate spread and is not useful

Leads to concept of variance and standard deviation

• Both variance and standard deviation are measures of how the data
vary about the mean

(Hughes & Hase)

Mean & standard deviation - 3…

12 March 2019 Experimental Work – Errors and presentation of results Slide 19

 Mean & standard deviation - 3
Variance = (Standard deviation)2
= sum of squares of deviations

For N points, Standard deviation − is given by

Distribution is normalised to (N-1) since data already used once

to get mean, so only (N-1) independent values of deviations to
determine variance

(Hughes & Hase)

Mean & standard deviation - 4…

12 March 2019 Experimental Work – Errors and presentation of results Slide 20

 Mean & standard deviation - 4

Mean & standard deviation - 5…

12 March 2019 Experimental Work – Errors and presentation of results Slide 21

 Mean & standard deviation - 5
For few measurements (e.g.3) – a quick method

(Hughes & Hase)

Reporting results – 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 22

 Reporting results - 1

• If error not quoted, assume it is in last digit

• The precision of a calculated result is limited by the least precise
measurement
• To avoid confusion with results ending in zero, quote them using
scientific notation

(Hughes & Hase) Reporting results – 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 23

 Reporting results – 2
Errors should be reported as
• absolute errors (X0 ± dX) or
• relative errors (X0 ± X%)

Example

Measure the length of box and a ladder using a tape measure with an
absolute error of 1 mm

Length of box = 300 mm so relative error = 1/300. = 0.33%

Length of ladder = 3.275 m so relative error = 0.001/3.275 = 0.0305%

Absolute errors equal, but relative errors very different

(Hughes & Hase) Reporting results - 3…

12 March 2019 Experimental Work – Errors and presentation of results Slide 24

 Reporting results - 3

In this case use a decimal point: 270 has two significant figures and 270.
has three significant figures

(Hughes & Hase) Reporting results - 43…

12 March 2019 Experimental Work – Errors and presentation of results Slide 26

 Reporting results - 4

Reporting results - 43…

12 March 2019 Experimental Work – Errors and presentation of results Slide 27

 Reporting results - 5

Reporting results - 6

12 March 2019 Experimental Work – Errors and presentation of results Slide 27

 Reporting results - 6

Reporting results - 7…

12 March 2019 Experimental Work – Errors and presentation of results Slide 28

 Reporting results - 7

Summary - reporting results - 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 29

 Summary: reporting results - 1
Five golden rules to obey when reporting experimentally
determined values

(Hughes & Hase)

Rejecting outliers - 1 …

12 March 2019 Experimental Work – Errors and presentation of results Slide 30

 Rejecting outliers - 1
Controversial practice – do carefully

One way – Chauvenet’s Criterion

• Test based on Gaussian distribution
• Aim to assess whether a point that lies many error bars from the
mean should be discarded (i.e. regarded as an outlier)

Rejecting outliers - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 31

 Rejecting outliers - 2

Apply procedure only once with care, but note that it may reject >
1 point
Try to determine reason for outlier
Justified if the expected distribution of points is known from
previous experiments or theoretical predictions
For a limited data set consider removing an outlier
Recalculate mean, std deviation etc.
Best way to resolve problem, if possible, is to repeat
measurement under same conditions (best solution for you!)

Propagation of errors - 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 32

 Propagation of errors - 2
Example of error calculation (A)
The diameter of a pipe of length L (error L) containing MW kg of
water (error MW & density W) is

So applying (A) the error in calculated D is then

Always check for dominant error(s)

• For example if Z = A x B x C x D and A is known to 5%, B, C & D are known to
1% then the error will effectively be 5% and there is no need to do the
analysis

Fitting function - 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 33

 Propagation of errors - 1

(Hughes & Hase) Propagation of errors - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 34

 Fitting functions - 1

Some brief examples

Fitting function - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 35

 Fitting function - 2
A silly fit (Harrison, 2001)

Fitting function - 3…

12 March 2019 Experimental Work – Errors and presentation of results Slide 37

 Fitting function - 3
Visual inspection may indicate a good fit to experimental data –
even so, plot residuals (deviations)
• If data consistent with fitted function, residuals should have zero
mean and no obvious trend

(Hughes & Hase) Fitting function - 4…

12 March 2019 Experimental Work – Errors and presentation of results Slide 37

 Fitting function - 4 (Haldenwang 2003)

Fitting function - 5…
12 March 2019 Experimental Work – Errors and presentation of results Slide 38
 Fitting function - 5

For examples and more information on residuals, see

www.itl.nist.gov/div898/handbook

Go to: “Chapters” or “Tools & Aids”

(On-line and printer friendly examples)

Presenting data - 1…

12 March 2019 Experimental Work – Errors and presentation of results Slide 39

 Presenting data - 1
Graphical representation is the most efficient way of reporting
experimental results

A good graph enables easy recognition of the key information

contained in the data

A good graph must

• Be simple
• Be clear
• Contain all the pertinent information

Presenting data - 2…

12 March 2019 Experimental Work – Errors and presentation of results Slide 41

 Presenting data - 2

(Hughes & Hase) Presenting data - 3…

12 March 2019 Experimental Work – Errors and presentation of results Slide 41

 Presenting data - 3

Presenting data - 4…

12 March 2019 Experimental Work – Errors and presentation of results Slide 41

 Presenting data - 4

Presenting data - 5…

12 March 2019 Experimental Work – Errors and presentation of results Slide 43

 Presenting data - 5

Prefixes table…

12 March 2019 Experimental Work – Errors and presentation of results Slide 44

 Prefixes table

References…

12 March 2019 Experimental Work – Errors and presentation of results Slide 45

 References
• Cottrill A. Regression Analysis: Basic Concepts.
• http://www.wfu.edu/~cottrell/ecn215/regress.pdf (1 Aug 2008)
• Fornasini, P. (2008) The Uncertainty in Physical Measurements. Springer
• Furlong, C. 2010. Engineering Experimentation ME-3901, D’2010. Lecture 03
• http://users.wpi.edu/~cfurlong/me3901/lect03/Lect03.pdf
• Groebner D.F. and Shannon P.W. (1985). Business Statistics, A Decision Making Approach. Charles E.
Merrill Publishing company, Ohio.
• Haldenwang, R., (2003), Flow of non-Newtonian fluids in open channels. Unpublished D.Tech thesis.
Cape Technikon, Cape Town
• Harrison D.M (2001). Error Analysis in Experimental Physical Science
• http://www.upscale.utoronto.ca/PVB/Harrison/ErrorAnalysis/ (1 Aug 2008)
• Holman, J.P. 2001. Experimental Methods for Engineers. McGraw-Hill International edition.
• Hughes, IG & Hase, TPA. 2010. Measurements and their uncertainties: a practical guide to modern error
analysis. Oxford University Press
• http://en.wikipedia.org/wiki/Error (1 Aug 2008)
• Jacobs B, http://www.uni-saarland.de/z-einr/MZ/graph/titel.html (1 Aug 2008)
• Ross, S.M. 2003. Peirce’s criterion for the elimination of suspect experimental data. Journal of
Engineering technology.
• Usher A. Errors, What they are and how to deal with them.
http://hug.phys.huji.ac.il/PHYS_HUG/MAABADA/mitkademet_a/errors.pdf
• http://www.rit.edu/cos/uphysics/uncertainties/Uncertaintiespart1.html
• http://www.rit.edu/cos/uphysics/uncertainties/Uncertaintiespart2.html
Glossary…

12 March 2019 Experimental Work – Errors and presentation of results Slide 47

 Glossary

12 March 2019 Experimental Work – Errors and presentation of results Slide 48