Surabaya, 24 September 2022

Understanding Levey-Jennings
and Westgard Rules in
Laboratory Quality Control

Fauqa Arinil Aulia, dr., Sp.PK

Indonesian Association of Clinical Chemistry (IACC)
Perhimpunan Dokter Spesialis Patologi Klinik (PDS PATKLIN)
“Without
laboratories
man of science
are soldiers
without arms”
Louis Pasteur
 01 Introduction 02 Levey-Jennings
Charts

03 Westgard Rules 04 Case Study

What we
discuss
today
01 Introduction
 Laboratory Error
• Laboratory error is defined as ➔
• any defect from ordering tests to reporting and
interpretation of results.
• Frequency = 0.012-0.6% of all test results
• Have huge impact ➔
• 60–70% of all diagnosis and patient management
are made on the basis of laboratory tests.

Laboratory Medicine, Volume 45, Issue 2, May 2014, Pages e80–e88, https://doi.org/10.1309/LMD0YIFPTOWZONAD 5
 Testing Process
• Three phases ➔
• Preanalytical:
• determining and ordering the test, identifying the
patient, collection and specimen transportation
to the laboratory
• Analytical:
• specimen preparation and testing
• Postanalytical:
• reporting result

Laboratory Medicine, Volume 45, Issue 2, May 2014, Pages e80–e88, https://doi.org/10.1309/LMD0YIFPTOWZONAD 6
 Surveys
• Most errors ➔ pre-analytical phase
• 46-68.2% of total errors
• A high error rate ➔ post-analytical phase
• 18.5-47% of total errors
• Significantly reduced ➔ analytical phase

Clin Chem Lab Med 2006;44(6):750-9. DOI: 10.1515/CCLM.2006.123 7

 Measurement Errors
• Never be truly nullified, but it can be decreased
to a scale that is acceptable.
• Random error:
• unpredictable variations in sample, instrument,
measurement process or analysis
• Systematic error:
• a form of measurement error that skews the results
to one side. Repeating the measurements cannot
eliminate it.

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 8

 Random Error
• Pola error tidak tetap.
• Menunjukkan tingkat presisi pemeriksaan.
• Contoh:
1. Gelembung pada reagen dan garis reagen
2. Pencampuran reagen yang inadekuat
3. Suhu yang tidak stabil dan inkubasi
4. Supply listrik yang tidak stabil
5. Variasi operator (pemipetan, waktu, dan lain-lain)

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 9

 Systematic Error
• Pola error tetap.
• Menunjukkan tingkat akurasi pemeriksaan.
• Contoh:
1. Pembuatan standar/reagen yang kurang baik
2. Instrumen yang kurang baik
3. Prosedur penulisan yang kurang tepat

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 10

 Laboratory Quality Control
• to detect, reduce, and correct deficiencies in a
laboratory's internal analytical process,
• to improve the quality of the results reported by
the laboratory.

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 11

02 Levey-Jennings
Charts
 Levey-Jennings Chart

• It is named after pathologist Stanley Levey

and E. R. Jennings in 1950,
• is a graph that quality control data is plotted on
• to give a visual indication whether a laboratory test
is working well
• to make easy to see how far off the result of
laboratory was
Levey, Stanley; Jennings, E. R. (November 1, 1950). "The use of Control Charts in the Clinical Laboratory". American Journal of
Clinical Pathology. 20 (11): 1059–1066. doi:10.1093/ajcp/20.11_ts.1059
13
 Levey-Jennings Chart
• The x-axis ➔
• the date and time, or more often the number of the control run
• The y-axis➔
• lines run across the graph at the mean, as well as one, two
and three standard deviations to either side of the mean
• each positive control should be compared with established
mean and standard deviations of the control sample

Levey, Stanley; Jennings, E. R. (November 1, 1950). "The use of Control Charts in the Clinical Laboratory". American Journal of
Clinical Pathology. 20 (11): 1059–1066. doi:10.1093/ajcp/20.11_ts.1059
14
 Developing
Levey-Jennings Chart
1. Select the number of data points (k) to be collected
before control limits are calculated (at least twenty).

2. Calculate the overall process average to measure

mean (if the control value is not being used for the
center line):

where k is the number of control samples run.

WHO Quality Management Handbook (2011): chapter 7, pages 81-88 15

 Developing
Levey-Jennings Chart
3. Calculate the standard deviation (if a known
standard deviation is not being used):

We can easily calculate this in Excel using the

STDEV function.

4. Calculate the upper control limit (UCL) and the

lower control limit (LCL).

WHO Quality Management Handbook (2011): chapter 7, pages 81-88 16

 Developing
Levey-Jennings Chart
5. Plot the center line as a sold line on the Levey-
Jennings chart and label.
6. Plot the control limits on the Levey-Jennings chart
and label.
7. If using the Westgard rules, plot the lines at +1s, +2s,
-1s and -2s from the center line.
8. Interpret the chart - what is it telling you about the lab
test?
WHO Quality Management Handbook (2011): chapter 7, pages 81-88 17
Levey, Stanley; Jennings, E. R. (November 1, 1950). "The use of Control Charts in the Clinical Laboratory". American Journal of
Clinical Pathology. 20 (11): 1059–1066. doi:10.1093/ajcp/20.11_ts.1059
18
19
03 Westgard Rules
 Deteksi Error
• Single-rule Quality Control
• Multi-rule Quality Control

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
21
 Single-rule QC
• Rules ➔
• 12s dan 13s
• Kelebihan:
• Lebih praktis
• Mudah dipahami
• Kekurangan:
• Angka false rejection tinggi (10-20%)
• Deteksi error rendah

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 22
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 Multi-rule QC
• Rules ➔
• Westgard Rules, yaitu 13s ; 22s ; R4s ; 41s ; 10x
• Kelebihan:
• Angka false rejection rendah (1%)
• Deteksi error tinggi
• Kekurangan:
• Lebih rumit dalam pengerjaan dan interpretasi

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 23
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 N = “Numbers”
• N (numbers) adalah jumlah total
pengukuran kontrol yang tersedia
• N = 2 → 2 pengukuran dengan 1 bahan control
atau 1 pengukuran dengan 2 bahan kontrol
• N = 3 → 1 pengukuran dengan 3 bahan kontrol
• N = 4 → 2 pengukuran dengan 2 bahan control
atau 4 pengukuran dengan 1 bahan kontrol

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
24
 Westgard Rules
• set by Dr. James Westgard
• a guidelines in laboratory quality control
experiments for identification of
• random error and within runs
• systematic error and between runs
• based on repeated measurements of at least
two reference samples with each analytical run

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical Chemistry", Clinical
Chemistry, vol. 27, pp. 493–501.
25
 Westgard Rules
• When any of these rules is violated, the process
behind the numbers is ‘out-of-control’ ➔ should be
stopped and investigated.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 26
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 Westgard Rules
• 13s ➔ One value beyond 3*sigma from the mean.
• 22s ➔ Two consecutive values either greater than, or
less than, 2*sigma from the mean.
• R4s ➔ A difference between consecutive values
greater than 4*sigma.
• 41s ➔ Four consecutive values greater than, or less
than, 1*sigma from the mean.
• 10x ➔ Ten consecutive values all greater than, or less
than, the mean.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 27
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
12s

Definisi:
Kriteria warning untuk pemeriksaan data kontrol secara hati-hati.

Solusi:
Lihat performa hasil kontrol lainnya, yaitu hasil kontrol sebelumnya dalam level
yang sama (across-run) dan hasil kontrol level lainnya pada saat dikerjakan
bersamaan (within-run).

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 28
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
13s

Definisi:
Kriteria rejected run bila hasil pengukuran 1 kontrol melebihi limit
kontrol mean +3SD atau –3SD.

Solusi:
Evaluasi penyebab random error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 29
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
22s

Definisi:
Kriteria rejected run bila:
• Dua hasil kontrol terakhir dari level kontrol yang sama, keluar di sisi yang
sama baik + 2 SD atau – 2SD (across-run)
• Dua hasil kontrol terakhir dari level kontrol yang berbeda, keluar di sisi yang
sama baik + 2 SD atau – 2SD (within-run)
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 30
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
R4s

Definisi:
Kriteria rejected run bila:
• Dua hasil kontrol terakhir dari level kontrol yang sama (across-run) atau
berbeda (within-run), keluar dari 2SD di sisi yang berseberangan sehingga
perbedaan nilainya menjadi 4SD
• Tiga level yang dikerjakan dan 2 hasil diantaranya berbeda 4SD
Solusi:
Evaluasi penyebab random error.
Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 31
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
41s

Definisi:
Kriteria rejected run bila:
• Empat hasil kontrol terakhir dari level kontrol yang sama
(across-run) atau berbeda (within-run), berada pada sisi yang
sama diatas nilai +1SD atau dibawah – 1SD.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 32
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
10x

Definisi:
Kriteria rejected run bila:
• Sepuluh hasil kontrol terakhir dari level kontrol yang sama
(across-run) atau berbeda (within-run), berada pada sisi yang
sama di atas atau di bawah nilai rata-rata.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 33
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 Westgard Rules
(Addition)
▪ Usually used with N's of 2 or 4
▪ Appropriate when two different control materials are
measured 1 or 2 times per material
• 8x - reject when 8 consecutive control measurements fall on
one side of the mean.
• 12x - reject when 12 consecutive control measurements fall on
one side of the mean

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 34
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 8x

Definisi:
Kriteria rejected run bila:
• Delapan pengukuran kontrol secara berturut-turut berada di
satu sisi mean.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 35
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
12x

Definisi:
Kriteria rejected run bila:
• Dua belas pengukuran kontrol secara berturut-turut berada di
satu sisi mean.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 36
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 Westgard Rules
(Other Common Multirules)
Where 3 different control materials are being analyzed:
• 2 of 32s - reject when 2 out of 3 control measurements exceed
the same mean plus 2s or mean minus 2s control limit
• 31s - reject when 3 consecutive control measurements exceed
the same mean plus 1s or mean minus 1s control limit.
• 6x - reject when 6 consecutive control measurements fall on one
side of the mean.
• 9x - reject when 9 consecutive control measurements fall on one
side of the mean.
• 7T - reject when seven control measurements trend in the same
direction, i.e., get progressively higher or progressively lower.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 37
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
2 of 32s

Definisi:
Kriteria rejected run bila:
• Dua dari tiga pengukuran kontrol melebihi sisi mean yang sama
dari +2s atau -2s.
Solusi:
Evaluasi penyebab random error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 38
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
31s

Definisi:
Kriteria rejected run bila:
• Tiga pengukuran kontrol berturut-turut melebihi mean yang
sama dari +1s atau -1s.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 39
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 6x

Definisi:
Kriteria rejected run bila:
• Enam pengukuran kontrol berturut-turut jatuh di satu sisi mean.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 40
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 9x

Definisi:
Kriteria rejected run bila:
• Sembilan pengukuran kontrol berturut-turut jatuh pada satu sisi
mean.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 41
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
 7T

Definisi:
Kriteria rejected run bila:
• Tujuh tren pengukuran kontrol dalam arah yang sama, yaitu,
semakin tinggi atau semakin rendah.
Solusi:
Evaluasi penyebab systematic error.

Westgard, J. O., P. L. Barry, and M. R. Hunt (1981). "A Multi-rule Shewhart Chart for Quality Control in Clinical 42
Chemistry", Clinical Chemistry, vol. 27, pp. 493–501.
43
44
04 Case Study
 PEMANTAPAN MUTU (QC) INTERNAL
KIMIA KLINIK
KONTROL : LEVEL 1 & 2
NO. LOT : 987654
Exp Date : Oktober 2022
BULAN : Agustus 2022

Parameter: LDL Parameter: LDL

Tanggal LEVEL 1 LEVEL 2 Tanggal LEVEL 1 LEVEL 2
X: 57.15 SD : 2.56 X: 147.71 SD: 5.34 X: 57.15 SD : 2.56 X: 147.71 SD: 5.34
1 56.7 -0.2 139.4 -1.6 16 63.2 2.36 151.1 0.63
2 56.3 -0.3 149.2 0.28 17 58.6 0.57 146.5 -0.2
3 57.1 -0 143.8 -0.7 18 55.5 -0.6 153.4 1.07
4 55.1 -0.8 141.8 -1.1 19 54.7 -1 152.7 0.93
5 55.7 -0.6 140.9 -1.3 20 54.5 -1 149.4 0.32
6 57 -0.1 151 0.62 21 54.7 -1 144.2 -0.7
7 57.1 -0 151.6 0.73 22 58.5 0.53 149.9 0.41
8 56 -0.4 148.1 0.07 23 57.8 0.25 142 -1.1
9 57.3 0.06 143.6 -0.8 24 60 1.11 152.1 0.82
10 58.5 0.53 142.4 -1 25 57.6 0.18 150.7 0.56
11 54.9 -0.9 147.1 -0.1 26 57.9 0.29 153.1 1.01
12 55.5 -0.6 135.7 -2.2 27 56.9 -0.1 150.3 0.49
13 59.5 0.92 143.6 -0.8 28 54.6 -1 147 -0.1
14 55.4 -0.7 149.6 0.35 29 56.9 -0.1 149.5 0.34
15 62.5 3.65 163.1 2.88 30 55.4 -0.7 151.6 0.73
31 56.4 -0.3 144.7 -0.6

46
 PEMANTAPAN MUTU (QC) INTERNAL
KIMIA KLINIK

KONTROL : Level 1 dan Level 2

NO. LOT : 987654
E.D : Oktober 2022
BULAN : Agustus 2022
Parameter: LDL
LEVEL 1
3
22s ➔ Rejected Run
2.5

1.5

1
LEVEL 1
0.5

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
-0.5

-1

-1.5

1 – 31 Agustus 2022
47
 22s ➔ Rejected Run

Systematic Error
• Trouble shooting:
1. Pembuatan standar/reagen yang kurang baik
2. Instrumen yang kurang baik
3. Prosedur penulisan yang kurang tepat

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 48

 PEMANTAPAN MUTU (QC) INTERNAL
KIMIA KLINIK

KONTROL : Level 1 dan Level 2

NO. LOT : 987654
E.D : Oktober 2022
BULAN : Agustus 2022
Parameter: LDL
LEVEL 2
4

2
12s ➔ Warning
1
LEVEL 2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-1

-2

-3 12s ➔ Warning
1 – 31 Agustus 2022
49
 12s ➔ Warning

Random Error
• Trouble shooting:
1. Gelembung pada reagen dan garis reagen
2. Pencampuran reagen yang inadekuat
3. Suhu yang tidak stabil dan inkubasi
4. Supply listrik yang tidak stabil
5. Variasi operator (pemipetan, waktu, dan lain-lain)

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 50

 PEMANTAPAN MUTU (QC) INTERNAL
KIMIA KLINIK

KONTROL : Level 1 dan Level 2

NO. LOT : 987654
E.D : Oktober 2022
BULAN : Agustus 2022
Parameter: LDL
22s ➔ Rejected Run
4

1
LEVEL 1
LEVEL 2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

-1

-2

-3 12s 12s ➔ Warning

1 – 31 Agustus 2022
51
 22s ➔ Rejected Run

Systematic Error
• Trouble shooting:
1. Pembuatan standar/reagen yang kurang baik
2. Instrumen yang kurang baik
3. Prosedur penulisan yang kurang tepat

Quality Control in Laboratory DOI: 10.5772/intechopen.72311 52

THANK YOU