You are on page 1of 18




Measurement Systems
Discrimination, stability, bias, repeatability,
reproducibility, and linearity
Modern measurement system analysis goes well beyond calibration. A gage
can be perfectly accurate when checking a standard and still be entirely unac-
ceptable for measuring a product or controlling a process. This section illus-
trates techniques for quantifying discrimination, stability, bias, repeatability,
reproducibility and variation for a measurement system. We also show how to
express measurement error relative to the product tolerance or the process var-
iation. For the most part, the methods shown here use control charts. Control
charts provide graphical portrayals of the measurement processes that enable
the analyst to detect special causes that numerical methods alone would not

Discrimination, sometimes called resolution, refers to the ability of the
measurement system to divide measurements into ‘‘data categories.’’ All
parts within a particular data category will measure the same. For example,

Inadequate gage discrimination on a control chart.1.000. 0. Figure 10. then a gage with a discrimination of 0.’’ This situation is illustrated in Figure 10..0002. if a process was capable (i.. A measurement system’s discrimination should enable it to divide the region of interest into many data categories. Six Sigma is less than the tolerance) and s ¼ 0:0005. but one with a discrimination of 0. then items measur- ing 1.0005 would be acceptable (six data categories).1. . i.e.e. When unacceptable discrimination exists. 1. the region of interest is the smaller of the tolerance (the high specification minus the low specifica- tion) or six standard deviations.9997 would all be placed in the data category 1.0003.326 MEASUREMENT SYSTEMS ANALYSIS if a measurement system has a resolution of 0. A measurement system should be able to divide the region of interest into at least five data categories.001 would not (three data categories). For example. they would all measure 1.000 inches with this particular measurement system. the range chart shows discrete ‘‘jumps’’ or ‘‘steps. In Six Sigma.001 inches.

Once statistical stability has been achieved. . the data plotted are the same. including variation from all causes. Control charts are then constructed and evaluated. say an hour. etc. the result is to make the X-bar chart go out of control. R&R studies for continuous data 327 Note that on the control charts shown in Figure 10.. repeatability. Statistical sta- bility is a broader term that refers to the overall consistency of measurements over time. including bias. which appears highly stratified. The effect is most easily seen on the R chart. measurement sys- tem stability can be determined. i. it is possible to adjust the control limits for the round-off error by using a more involved method of computing the control limits. the system either is or is not statistically stable. say weekly. even though the process is in control. If this cannot be done. see Pyzdek (1992a. The standard is measured repeatedly over a short time.1. STABILITY Measurement system stability is the change in bias over time when using a measurement system to measure a given master part or standard. sampling intervals and measurement procedures to be fol- lowed. add more significant digits. R chart method: R ^ ¼ d2 s chart method: s ^ ¼ c4 The values d2 and c4 are constants from Table 11 in the Appendix. One measure is the process standard deviation based on the R or s chart. 37^42) for details. Averages and range charts are typically plotted on mea- surements of a standard or a master part. No ‘‘stability number’’ is calculated for statistical stability. as shown by the control charts with unrounded data. A system’s statistical stability is determined through the use of control charts. reproducibility. Subject matter expertise is needed to determine the subgroup size. then the measurements are repeated at predeter- mined intervals. A (statistically) stable system will show no out-of-control signals on an X-control chart of the averages’ readings.e. The remedy is to use a measurement system cap- able of additional discrimination. but not before. pp. except that the data on the bottom two charts were rounded to the nearest 25. As sometimes happens (but not always).

425 25.425 25. then obtaining multiple measurements on the standard. The readings obtained are: 25. The appraiser then obtains a number of repeated measure- ments on the reference part.425 25.4 mm is checked 10 times by one mechanical inspector using a dial caliper with a resolution of 0. 254:051 X ¼ ¼ 25:4051 mm 10 The bias is the average minus the reference value. For example.25 mm then % bias ¼ 100  jbiasj=tolerance ¼ 100  0:0051=0:5 ¼ 1% .400 25. Since parts and processes vary over a range.025 mm.400 25.375 25. Estimating bias involves identifying a standard to represent the reference value.e. The standard might be a master part whose value has been determined by a mea- surement system with much less error than the system under study. Example of computing bias A standard with a known value of 25. Bias can be determined by selecting a single appraiser and a single reference part or standard.400 25. i.375 The average is found by adding the 10 measurements together and dividing by 10.400 25.. Bias is then estimated as the difference between the average of the repeated measurement and the known value of the reference part or standard. if this mea- surement system were to be used on a process with a tolerance of  0. bias will not be the same at each point in the range (see the definition of linearity above). bias ¼ average  reference value ¼ 25:4051 mm  25:400 mm ¼ 0:0051 mm The bias of the measurement system can be stated as a percentage of the toler- ance or as a percentage of the process variation. bias is measured at a point within the range. or by a stan- dard traceable to NIST.425 25.328 MEASUREMENT SYSTEMS ANALYSIS BIAS Bias is the difference between an observed average measurement result and a reference value. If the gage is non-linear.

If the range or sigma chart is in control then repeatability can be estimated by finding the standard deviation based on either the average range or the aver- age standard deviation. Bias example illustrated. The equations used to estimate sigma are shown in Chapter 9. pro- duce results that are 0. This difference represents 1% of the allowable product variation.1 m-inches. each part was checked twice by each inspector. R&R studies for continuous data 329 This is interpreted as follows: this measurement system will.2.2. whichever is greater. The gage has a resolution of 0. Figure 10. If the range or sigma chart is out of control. The situation is illustrated in Figure 10. Example of estimating repeatability The data in Table 10. The gage records the surface roughness in m- inches (micro-inches).0051 mm larger than the actual value. . REPEATABILITY A measurement system is repeatable if its variability is consistent. Each inspector checked the surface finish of five parts. then special causes are making the measurement system inconsis- tent. Consistent variability is operationalized by constructing a range or sigma chart based on repeated measurements of parts that cover a significant portion of the process variation or the tolerance. on average.1 are from a measurement study involving two inspec- tors.

1 108.0 119.35 0.6 124.15 1.4 108.3 4 118.4 4 120.9 124.75 0.7 130.6 118.40 0. Measurement system repeatability study data.2 124.3 119.1 5 130.3 112.0 3 124.330 MEASUREMENT SYSTEMS ANALYSIS Table 10. PART READING #1 READING #2 AVERAGE RANGE INSPECTOR #1 1 111.65 0.9 112.7 118.05 0.7 INSPECTOR #2 1 111.1 108.10 0.25 0.5 2 107.0 130.4 2 108.4 130.1.3 We compute: Ranges chart R ¼ 0:51 UCL ¼ D4 R ¼ 3:267  0:51 ¼ 1:67 Averages chart X ¼ 118:85 LCL ¼ X  A2 R ¼ 118:85  1:88  0:109 ¼ 118:65 UCL ¼ X þ A R ¼ 118:85 þ 1:88  0:109 ¼ 119:05 2 .9 112.6 124.4 112.7 5 130.10 0.1 130.7 3 124.7 108.65 0.

. Because the R chart is in control we can now estimate the standard deviation for repeatability or gage variation: R e ¼ ð10:1Þ d2 where d2 is obtained from Table 13 in the Appendix. and g is the number of parts times the number of inspectors (g ¼ 5  2 ¼ 10 for the example). This is the way it should be! Consider that the spread of the X-bar chart’s control limits is based on the average range. If the averages were within the control limits it would mean that the part-to-part varia- tion was less than the variation due to gage repeatability error. Note that we are using d2 and not d2 . This gives.3. an undesirable situation. which is based on the repeatability error. there are no special causes of variation. Table 13 is indexed by two values: m is the number of repeat readings taken (m ¼ 2 for the example). R&R studies for continuous data 331 The data and control limits are displayed in Figure 10. i.3. Figure 10. This indicates that the measurement system’s variability is consistent. Note that many of the averages are outside of the control limits.e. The d2 values are adjusted for the small number of subgroups typi- cally involved in gage R&R studies. Repeatability control charts. The R chart analysis shows that all of the R values are less than the upper control limit. for our example R 0:51 e ¼  ¼ ¼ 0:44 d2 1:16 .

each inspector’s average is computed and we find: Inspector #1 average ¼ 118:79 -inches Inspector #2 average ¼ 118:90 -inches Range ¼ Ro ¼ 0:11 -inches Looking in Table 13 in the Appendix for one subgroup of two appraisers we find d2 ¼ 1:41 ðm ¼ 2.15 is the Z ordinate which includes 99% of a standard normal distribution. Thus. Appraiser-to-appraiser variation represents a bias due to appraisers. This estimate involves averaging the results for each inspector over all of the readings for that inspector. this reproducibility estimate includes variation due to repeatability error. When this occurs the estimated reproducibility is set equal to zero. since each inspector checked each part repeatedly. The standard deviation of reproducibility (o ) is estimated by finding the range between appraisers (Ro) and dividing by d2 . can be estimated by com- paring each appraiser’s average with that of the other appraisers. Using these results we find Ro =d2 ¼ 0:11=1:41 ¼ 0:078. The appraiser bias. The reproducibility estimate can be adjusted using the following equa- tion: ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi s  ffi sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi   Ro 2 ð5:15e Þ2 0:11 2 ð5:15  0:44Þ2 5:15   ¼ 5:15   d2 nr 1:41 52 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ 0:16  0:51 ¼ 0 As sometimes happens. Reproducibility is then computed as 5. Reproducibility example (AIAG method) Using the data shown in the previous example.332 MEASUREMENT SYSTEMS ANALYSIS The repeatability from this study is calculated by 5:15e ¼ 5:15 0:44 ¼ 2:26. The value 5. the estimated variance from reproducibility exceeds the estimated variance of repeatability + reproducibility. we estimate that the reproducibility is zero. since negative variances are the- oretically impossible. g ¼ 1). However. REPRODUCIBILITY A measurement system is reproducible when different appraisers produce consistent results.15o . since there is only one range calculation g ¼ 1. or reproducibility. .

1 130.9 112.575 0.4 5 130 130.3 0.9 124.7 3 124.3 119.845 1 Observe that when the data are arranged in this way. the smallest reading for part #3 was from inspector #2 (124.7 Averages ! 118.2 124.7 120 119.7 108.4 130. For our data gage R&R ¼ 5:15  0:44 ¼ 2:27. R&R studies for continuous data 333 The measurement system standard deviation is pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi m ¼ e2 þ o2 ¼ ð0:44Þ2 þ 0 ¼ 0:44 ð10:2Þ and the measurement system variation.9).5 2 108.7 4 118.2.7 130. is 5. Thus. Reproducibility example (alternative method) One problem with the above method of evaluating reproducibility error is that it does not produce a control chart to assist the analyst with the evaluation. or gage R&R.4 112. INSPECTOR #1 INSPECTOR #2 Part Reading 1 Reading 2 Reading 1 Reading 2 X bar R 1 111.9 112. Table 10.125 1.2. For example.15m . Measurement error data for reproducibility evaluation.075 0.3 111.6 124.15 1.6 118.2) and the largest was from inspector #1 (124.6 124. This is shown in Table 10.1 107.4 108. the R value measures the combined range of repeat readings plus appraisers. This method begins by rearranging the data in Table 10. .1 108.1 so that all readings for any given part become a single row. The method presented here does this. R represents two sources of measurement error: repeatability and reproducibility.

Now we know that variances are additive. Averages chart X ¼ 118:85 LCL ¼ X  A2 R ¼ 118:85  0:729  1 ¼ 118:12 UCL ¼ X þ A R ¼ 118:85 þ 0:729  1 ¼ 119:58 2 The data and control limits are displayed in Figure 10. Figure 10. Using this method. The R chart analysis shows that all of the R values are less than the upper control limit. This indicates that the measurement system’s variability due to the combination of repeatabil- ity and reproducibility is consistent.4.334 MEASUREMENT SYSTEMS ANALYSIS The control limits are calculated as follows: Ranges chart R ¼ 1:00 UCL ¼ D4 R ¼ 2:282  1:00 ¼ 2:282 Note that the subgroup size is 4. we can also estimate the standard deviation of repro- ducibility plus repeatability.4. as we can find o ¼ Ro =d2 ¼ 1=2:08 ¼ 0:48. there are no special causes of variation. so 2 2 2 repeatabilityþreproducibility ¼ repeatability þ reproducibility ð10:3Þ . Reproducibility control charts..e. i.

Find the range of the part averages. If the part-to-part standard deviation is to be estimated from the measurement system study data. The total process standard deviation is found as t ¼ m2 þ p2 . To repeat. The 99% spread due to part-to-part variation (PV) is found as 5. Once the above calculations have been made. Rp. 2. most of the parts will fall outside of the X -bar chart con- trol limits. Where m ¼ the standard deviation due to measurement error. linear (see below). Compute p ¼ Rp =d2 . and consistent with respect to repeatability and reproducibility. then the measurement system is not capable of detecting normal part-to-part vari- ation for this process. Part-to-part variation can be estimated once the measurement process is shown to have adequate discrimination and to be stable. . 3. 5. If fewer than half of the parts are beyond the control limits. accurate. if the measure- ment system is adequate. the following procedures are followed: 1. 4. If not.15 p. 2. Substituting these values gives qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 reproducibility ¼ repeatabilityþreproducibility 2  repeatability qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ ð0:48Þ2  ð0:44Þ2 ¼ 0:19 Using this we estimate reproducibility as 5:15  0:19 ¼ 1:00. we computed repeatability ¼ 0:44. Con¢rm that at least 50% of the averages fall outside the control limits.15t . Plot the average for each part (across all appraisers) on an averages con- trol chart. PART-TO-PART VARIATION The X-bar charts show the part-to-part variation. The percent repeatability and reproducibility (R&R) is 100  ðm =t Þ%. 3. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1. the part-to-part standard deviation. the overall measurement sys- tem can be evaluated. The value of d2 is found in Table 13 in the Appendix using m ¼ the number of parts and g ¼ 1. since there is only one R calculation. ¢nd a better measurement system for this process. as shown in the reproducibility error alternate method. R&R studies for continuous data 335 which implies that qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi reproducibility ¼ 2 repeatabilityþreproducibility 2  repeatability In a previous example. Total variability (TV) is 5.

5. Once the above calculations have been made. see Figure 10. If not. the overall measurement sys- tem can be evaluated. The total process standard deviation is found as t ¼ m2 þ p2 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffi t ¼ m2 þ p2 ¼ ð0:44Þ2 þ ð8:96Þ2 ¼ 80:5 ¼ 8:97 2. Con¢rm that at least 50% of the averages fall outside the control limits. Rp ¼ 130:3  108:075 ¼ 22:23. ¢nd a better measurement system for this process. 46:15 1:41  ¼ 28:67 ¼ 28 2:27 . 5:15  8:96 ¼ PV ¼ 46:15. The number of distinct data categories that can be created with this mea- surement system is 1:41  ðPV=R&RÞ. the part-to-part standard deviation. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1. The value of d2 is found in Table 13 in the Appendix using m ¼ the number of parts and g ¼ 1. The number of distinct data categories that can be created with this mea- surement system is 1. g ¼ 1. 5:15  8:97 ¼ 46:20 3. m ¼ 5.15t . The 99% spread due to part-to-part variation (PV) is found as 5.3. Rp. the measurement system error is acceptable.15p . 2. Compute p ¼ Rp =d2 . p ¼ 22:23=2:48 ¼ 8:96. 3. The percent R&R is 100  ðm =t Þ% m 0:44 100 % ¼ 100 ¼ 4:91% t 8:97 4. 4.336 MEASUREMENT SYSTEMS ANALYSIS 4. Done above.41  (PV/R&R). or 80%. d2 ¼ 2:48. are outside of the control limits. since there is only one R calculation. EXAMPLE OF MEASUREMENT SYSTEM ANALYSIS SUMMARY 1. Plot the average for each part (across all appraisers) on an averages con- trol chart. Thus. as shown in the reproducibility error alternate method. Find the range of the part averages. Total variability (TV) is 5. 4 of the 5 part averages.

Use gage R&R nested when each part can be measured by only one operator. To illustrate these capabilities. select Stat > Quality Tools > Gage R&R Study (Crossed) to reach the Minitab dialog box for our analysis (Figure 10. For reference purposes. choose gage R&R crossed. To do this. Minitab also offers both the .5). Gage R&R analysis using Minitab Minitab has a built-in capability to perform gage repeatability and reproduci- bility studies. the analysis indicates that this measurement system is more than adequate for process analysis or process control. To begin. In addition to choosing whether the study is crossed or nested. Data formatted for Minitab input. the previous analysis will be repeated using Minitab. Otherwise. the data must be rearranged into the format expected by Minitab (Figure 10. as with destructive testing.5. Figure 10. columns C1^C4 contain the data in our original format and columns C5^C8 contain the same data in Minitab’s preferred format. R&R studies for continuous data 337 Since the minimum number of categories is five.6). Minitab offers two different methods for performing gage R&R studies: crossed and nested.

This is useful informa- tion if the gage is to be used to make product acceptance decisions. There is an option in gage R&R to include the process tolerance. For example.338 MEASUREMENT SYSTEMS ANALYSIS Figure 10. etc. For the purposes of illustration. Much of the output is identical to what has been discussed earlier in this chapter and won’t be shown here. a gage capable of 5 distinct data categories for the process may have 4 or fewer for the product. if the process is not capable. multiple tables. If the process is ‘‘capable’’ in the sense that the total variability is less than the toler- ance. If the ANOVA method is selected. Note that the results of the calculations will differ slightly from those we obtained using the X-bar and R methods. then any gage that meets the criteria for checking the process can also be used for product acceptance. ANOVA and the X-bar and R methods. Minitab still displays the X-bar and R charts so you won’t lose the information contained in the graphics. then its out- put will need to be sorted and the gage used for sorting may need more discrimi- natory power than the gage used for process control. Output Minitab produces copious output. including six separate graphs. .7). we entered a value of 40 in the process tolerance box in the Minitab options dialog box (Figure 10.6. However. We will use ANOVA in this example. Minitab gage R&R (crossed) dialog box. This will provide comparisons of gage variation with respect to the specifications in addi- tion to the variability with respect to process variation. You must choose the ANOVA option to obtain a breakdown of reproducibility by operator and operator by part.

0.12 0. Minitab gage R&R (crossed) options dialog box.3 shows the analysis of variance for the R&R study. The F-ratios are used to compute the P values. R&R studies for continuous data 339 Figure 10.27.269/0.269 1.212 = 1. which show the probability that the observed var- iation for the source row might be due to chance.27 0.061 0.212 Total 19 1304.6602 Operator*PartNum 4 1.15 0 Operator 1 0. Two-way ANOVA table with interaction.06 0. The F-ratio for the Operator effect is found by using the Operator*PartNum interaction MS term as the denominator.22 0.43 .061/0. a P value less than 0.18 325. 0. In the ANOVA the MS for repeatability (0.08 0.34317 Repeatability 10 2.22.05 is the critical value for deciding that a source of variation is ‘‘signifi- Table 10.269 = 0. By convention.7. Table 10.294 1208. Source DF SS MS F P PartNum 4 1301.3.212) is used as the denominator or error term for cal- culating the F-ratio of the Operator*PartNum interaction.

For example.19 0. Minitab’s next output is shown in Table 10. Table 10. Two-way ANOVA table without interaction.05 so we conclude that the differences accounted for by these sources might be zero. indicating that the part-to-part variation is almost certainly not zero. The variance analysis shown in Table 10. This analysis has removed the interaction term from the model. In some cases this might identify a significant effect that was missed by the larger model.34) are greater than 0.294 1426.06 0. The P values for Operator (0.05) we would conclude that there were statistically significant differences between operators. while the % of VarComp shows the percentage of the total variance accounted for by each source.6 shows the spread attributable to the . If the Operator term was significant (P < 0.) Technically.. we would conclude that one operator has obtained differ- ent results with some. the P value for the PartNum row is 0.66) and the Operator*PartNum interaction (0.43 Minitab also decomposes the total variance into components.061 0. The analysis indicates that nearly all of the variation is between parts.73 0 Operator 1 0.4.5. The VarComp column shows the variance attributed to each source. while accurate. If the interaction term was significant. (Variances are the squares of measurements. greater than zero. while dispersion measurements expressed in original units are not. but not all. Source DF SS MS F P PartNum 4 1301.e.27 0.’’ i. this is the cor- rect way to analyze information on dispersion because variances are additive. is not in original units.5.4.228 Total 19 1304. prompting an investigation into underlying causes.18 325. but for this example the conclusions are unchanged.340 MEASUREMENT SYSTEMS ANALYSIS cant. parts. thereby gaining 4 degrees of freedom for the error term and making the test more sensitive. there is a natural interest in seeing an analysis of dispersion in the original units so Minitab provides this. Table 10. as shown in Table 10. However.6145 Repeatability 14 3.

whichever is greater. Bias is determined at each point in the range and a linear regression analysis is performed.228 0. This indicates that the measured process spread exceeds the tolerance.495 100 different sources.228 0.6 is presented graphically in Figure 10. The Study Var column shows the 99% confidence interval using the StdDev. Although this isn’t a process capability analysis.8. pub- lished by the Automotive Industry Action Group. The % Study Var column is the Study Var column divided by the total variation due to all sources. And the % Tolerance is the Study Var column divided by the tolerance. The StdDev column is the standard deviation. Linearity Linearity can be determined by choosing parts or standards that cover all or most of the operating range of the measurement instrument.28 Reproducibility 0 0 Operator 0 0 Part-to-Part 81. A scatter diagram should also be plotted from the data.28 Repeatability 0. Linearity is defined as the slope times the process variance or the slope times the tolerance.72 Total Variation 81. .5. It is interesting that the % Tolerance column total is greater than 100%. LINEARITY EXAMPLE The following example is taken from Measurement Systems Analysis. the data do indicate a possible problem meeting tolerances.267 99.5. The information in Table 10. or the square root of the VarComp column in Table 10. Components of variance analysis. R&R studies for continuous data 341 Table 10. Source VarComp % of VarComp Total gage R&R 0.

8.23 Figure 10.4262 99.4591 5.15 Reproducibility 0 0 0 0 Operator 0 0 0 0 Part-to-Part 9.02743 46.86 116.29 6.15*SD) (%SV) (SV/Toler) Total gage R&R 0.4913 100 116. Analysis of spreads. Study % Var % Study Var Tolerance Source StdDev (5.342 MEASUREMENT SYSTEMS ANALYSIS Table 10.29 6. .4591 5.47749 2.6.15 Repeatability 0.0148 46. Graphical analysis of components of variation.47749 2.07 Total Variation 9.