Avoid misunderstandings and mismanagement of statistical limits in manufacturing settings

In 50 Words Or Less
Types of limits include specication, control, disposition and reasonableness. Many individuals and organizations have difculty distinguishing between limits, leading to mismanagement of product and process. Knowing how to apply limits in manufacturing settings will help ensure product functionality, effective process control and intelligent business decisions.

Know Your Limits

by Robert K. Henderson

STATISTICS

IN MORE THAn 27 years of working with a

variety of manufacturing concerns, it is still surprising to observe how many manufacturing operations peoplefrom managers to engineers to technicians possess such limited understanding of the myriad limits they adhere to each and every day. These limits, applied to numerical measurements of key product quality characteristics, often determine and drive their behavior and actions. But these limits often can be viewed as frustrating hurdles to the successful execution of tasks rather than helpful guides to effectively manufacturing product suitable for sale to customers. This lack of understanding of various limits, often on an organizational scale, can contribute signicantly to the level of frustration surrounding their applications. Operations can become dysfunctional when this lack of understanding produces additional limits and further tightening of limitsoften by seemingly arbitrarily determined amountsby those with the power to implement such decisions in an organization.

May 2011 QP 37

Moreover, in the age of automationin which the number of manufacturing parameters routinely measured can range from the tens of thousands to more than 1 million each daywasted time and resources incurred through poor understanding and mismanagement of limits can be signicant.

individual quality measurement. To the engineer, if a product meets spec, it must be fully functional for its intended end use. To manufacturing personnel, if the product meets spec, it is assumed to be shippable to customers and can be added to production totalsalways a desirable outcome. Of the three or four sets of limits relevant to any measured quality characteristic, however, specication limits often will be the least important to actual manufacturing operations and decision-making processes. Of course, this assumes these specication limits are properly dened, understood and managed, which is a highly questionable assumption, even for large and well-known multinational manufacturing organizations. What are specication limits? The denition generally provided is limits within which an individual product unit would be expected to perform as intended for a suitable array of customer end uses. Consequently, specication limits are related to product design. Ideally, they are set in the product design steps and effectively xed for any subsequent manufacture of the product. Unfortunately, many of organizations do not recognize this denition of specication limits. Perhaps the most obvious abuse of the generally accepted denition involves one large and well-known organization applying specication limits to averages of process results, thereby completely failing to acknowledge the application of specication limits to individual product units. Imagine its surprise and horror when customers began requesting Cpk results, and its calculations invariably produced Cpk values less (and often much less) than one. The organization failed to understand that the variation of average results used to set its specication limits would be signicantly less than the variation of individual product unit measurements the customer required for use in calculating Cpk. The organization as a wholeincluding its large quality groupfailed to grasp a necessary understanding of the central limit theorem, arguably one of the most important theorems in statistics. If properly dened specications are really of limited value to manufacturing operational decision making, then what limits are valuable? Table 1 shows there are primarily twoand often threesets of important limits. One set has already been noted: control limits. The other set is disposition limits, often given other monikers, such as release limits, scrap limits or rework limits.

Specication limits
While explanations of the differences between control limits and specication limits have been widely disseminated for many years, these differences seem to be poorly understood at all levels of many organizations, including, unfortunately, quality professionals. Most engineers and manufacturing personnelfrom technicians to managersstill focus on specication limits as the most important limits associated with any

Limits applied to measurements of key product quality characteristics / taBLE 1

Aspect Disposition limits To dene a range To dene a range of values within of values within which a specic To dene a range of which a specic statistic must fall values within which statistic must fall to conclude that an individual product to conclude that a certain amount unit will satisfactorily the associated of processed perform for expected process is in product can be customer end uses control (on further processed target) and shipped to a customer Product functionality, reporting to Process control Product disposition customers and management Balance between Minimize overall false signal rate Product design (producer and and desired customer) costs sensitivity All future product units A specic, nite All product units produced under group of processed (past and future) current process product units conditions Low High High Specication limits Control limits

Purpose

Use

How determined

Applicability

Frequency of use in manufacturing operations Primary action when not met Relative width

Scrap or rework product unit Widest

Shut down Scrap, rework process to or designate investigate issue associated group or adjust process of product units for back to target further evaluation Narrowest Between

38 QP www.qualityprogress.com

STATISTICS

The organization mentioned earlier actually manages its specication limits as disposition limits, so it has not yet fully recognized the need for separate sets. The third set of limits that often is useful is a set of reasonablenessier or outlierlimits. Even with automated data collection, it is possible for obviously erroneous results to enter the data stream. Therefore, such limits can have value in effectively avoiding inaccurate assessments of product and process conditions.

In the semiconductor industry, for example, the product unit is a dieor chipthat is produced on wafers within lots comprised of a specic number of wafers. The structure of the production process generates many potential sources of inherent variationfor example, die-to-die within wafer, wafer-to-wafer within lot and lotto-lot within production run. Most sampling in this industry, however, is done by measuring several dice on a specic wafer within a specic lot, with the resultant wafer average being used as the control statistic. While this is a logistically and likely an economically appropriate sampling approach, it fails to produce the rational subgroups desired for common SPC applications. There still will be inherent, commoncause variation across sample groups. As a result, simply using a common estimate of within-group variation produces an underestimate and sometimes a gross underestimate of the true inherent variation in the process. This will generate control limits that are too tight, resulting in over control or, ultimately, so many false out-of-control results (signals) that all signals begin to be ignoredeven any potential real signalsand the control limits become, in effect, useless. The remedial action is, of course, to acquire the data to fully understand the nature of the inherent, in-control process variation and set control limits based on these results. Unfortunately, most SPC courses and many standard references do not explain the necessity of analysis of variance (ANOVA) or details of how to perform ANOVA properly to extract valid estimates of the relevant process variance components required to effectively estimate common-cause process variation in these often-encountered processes. Another issue in establishing effective control limits is related to a concept mentioned earlier: a manageable false signal rate. Shewhart and Deming were proponents of limits using a multiplier of three times an appropriate estimate of the standard deviation of the control statistic.2, 3 With no error in such an estimate and no deviation of the process from operating at the center of the control limit window, this would result in about one false signal in 370 sampling periods. When Shewhart introduced this approach in the 1920s, and even when Deming helped the Japanese successfully implement it in the 1980s, the computational capability we have today was not yet available. As a result, control charts were managed by hand.

Control limits
There have been many publications, courses and discussions related to the concept of control limits, initially attributed to Walter Shewhart and his work at Bell Labs in the 1920s.1 The general idea of control limits is to periodically sample the process, measure a quality characteristic onideallyseveral product units, and then plot these resultsgenerally, the average of the resultsthrough time. With a sufcient process history obtained, process control limits for future sample results can be determined. Any results outside these control limits would indicate the process was not behaving as desiredin other words, was out of control. When the data suggest the process is in such an undesirable state, the implication is to stop the process for operators to investigate and rectify the cause of the issue, or to manage it in some other fashion (for example, adjust it in some manner) to return the process to the desired state of operation. This is an inherently simple concept and is likely one reasonwith apologies to W. Edwards Deming and the Japanese success in the 1980sstatistical process control (SPC) has become so popular and widespread in industry today. As simple as the concept seems, however, effective implementation remains elusive to many organizations. Many businesses struggle with the concept of rational subgroups or choosing an effective sampling period and strategy. Often, sampling strategy is determined by logistics within an organizations manufacturing facilities and processes. Frequently, the logistics do not produce what is desired of rational subgroups. Ideally, rational subgroups will be groups of product that capture or include virtually all of the inherent natural, or common cause, variation for the quality characteristic of interest. This may be possible in some industries, but in many businesses, this is not so easily obtained.

May 2011 QP 39

Fewer of these charts were being used in manufacturing environments, and only the ostensibly important product and process characteristics were being charted and controlled. Manufacturers could afford a false signal rate of one in 370 observations, which would be about one false signal a year for a chart with daily sampling. In many of todays manufacturing operationswith widespread automated data collection, storage and processing capabilitiesit is not uncommon for 50,000 to more than 1 million chartable quality-related measurements to be obtained in a 24-hour period. If all of these are charted with accurate estimates of related variation and control limits calculated using a multiplier of three, theneven if none of them ever deviated from operating at the center of their respective control limit windowssuch operations could expect about 135 to 2,700 false signals per day.

tions confuse these two types of limits and nd it difcult to distinguish them. Disposition limits are focused on product control. The decisions they drive are focused on what to do with an existing group of product that has already been processed through a specic process step or set of process steps. The basic decision involved is whether a specic group of product should be allowed to pass to further processing and eventually be shipped to customers. A result within suitably determined disposition limits would suggest additional processing is warranted, while a result outside these limits would indicate the product should be managed in a different mannerfor example, reworked, scrapped or tagged for special end-of-line evaluations. Disposition limits are different from control limits in at least three ways: 1. Application. Disposition limits are applied to a nite group of already-processed product. Control limits are applied to the future operation of a process for generally variable amounts of time and processed product. 2. Focus. Disposition limits are focused on product control and are ideally determined to minimize overall producer and customer costs. Control limits are focused on process control and are ideally determined by appropriately balancing false signal rates with required levels of sensitivity. 3. Risk. Disposition and control limits differ in the amount of risk, or perceived risk, related to the decisions they impose on a manufacturing operation. The risk associated with determining the fate of a nite amount of already-processed product (something has to be done with it) is often perceived as much less than the risk of determining the fate of the associated process. Adjusting a process will potentially impact all future product processed through the affected process step. Shutting down the step in the process for an investigative effort taking an indeterminate amount of time is rarely a popular decision in a manufacturing operation. Due to the inherent differences between product disposition and process control, it is desirable to separate these functions as much as possible. Some organizations frequently fail to do this and create situations in which

Disposition limits
In addition to process control limits, product disposition limits are arguably just as important to manufacturing operations. Their purpose is signicantly different from the purpose of process control limits, but many organiza-

Specication, disposition and control limits relationship / FIGurE 1

Oxide thickness example
Sampling distribution of EWMA: The variance of this distribution is approximately (0.2/(2-0.2)) = 0.111 times the variance of the lot averages. Control limits apply for this distribution @ ~ 10A Sampling distribution of lot averages: Due to the fixed sampling scheme within wafers, none of the wafer profile variation is present in this distribution. Random within-wafer variation is lower due to averaging. Disposition limits apply for this distribution @ ~ 20A Semiconductor process Variable: oxide thickness (A) Product unit: die Sampling: 1 wafer/lot 5 measurements at top, left, center, right, bottom (fixed) Disposition statistic: lot averages Control statistic: EWMA (0.2) of lot averages Product unit distribution: Mixture of low center of wafer and higher edge of wafer die. Specification limits apply for this distribution @ ~ 40A

540

550

560 Die

570

580

590

600

610

the statistics used to drive both functions are not only the same value, but also have identical limits. While it is possible to make such scenarios work, it

Lot average

EWMA

EWMA = exponentially weighted moving average

40 QP www.qualityprogress.com

STATISTICS

often leads to one function dominating the other. Then, it is the process control function that suffers because the product control decision is necessary and carries less inherent and perceived risk. To avoid this potential for dysfunctional systems, it is desirable to decouple the two functions by using different limits or, more desirably, using different statistics to drive the decision making for each function. This is when a sequential control statistic (for example, a run rule, an exponentially weighted moving average or a cumulative sum) can be used for process control, while the periodic sample average can be used for product disposition purposes. These are related statistics, but not the same, and therefore will almost certainly have different sets of limits applied to them (see Figure 1 for an example). It is doubtful Deming would eagerly support the use of sequential process control statistics as a way to decouple the process control and product disposition functions. He probably would be more inclined to advocate eliminating the product disposition function entirely and support reliance on simple process control approaches. Most manufacturing managers, however, are reluctant to discard the perceived safety net that disposition limits provide. Disposition limits (even if misnamed and misunderstood) are likely to always be part of any manufacturing operation.

cess with or without additional qualication requirements. Reasonableness limits drive decisions on data management, allowing valid results into the data stream for decision-making purposes and excluding erroneous measurements or replacing them with re-measurements. If specication limits do not play a direct role in any of the earlier decisions, then whats their value? While they are indeed of limited value for the daily decision making within the manufacturing environment, specication limits can facilitate determination of useful product disposition limits and may play a role in determining required sensitivity levels in setting process control limits. Specication limits, however, are primarily for interactions with customers and management. They are used in the calculation of Cpk statistics commonly shared with customers and periodically used by managersrightly or wronglyto evaluate how well different processes are managed. While this alone makes them important and of longterm value to manufacturing operations, specication limits are not generally thought to be of signicant value in routine, daily manufacturing endeavors. Do you or, more generally, your organization know your limits? A simple test would be to compare the respective limits in your organization with the ones included in this article. If these are not present and many additional limits beyond these are actively being used, it is likely your organization does not sufciently understand its limits. Perhaps you work for a manufacturer that effectively knows and manages the limits it applies to routinely acquired measurements of key product quality characteristics. If so, you are truly fortunate. Many organizations continue to operate in relative degrees of ignorance related to their routinely used limits and, as a result, endure daily frustrations. QP
REFERENCES
1. Walter A. Shewhart, Economic Control of Quality of Manufactured Product, D. Van Nostrand, 1931. 2. Ibid. 3. W. Edwards Deming, Out of the Crisis, MIT Press, 1986. ROBERT K. HENDERSON is an assistant professor of mathematics and statistics at Stephen F. Austin State University in Nacogdoches, TX. He holds a doctorate in mathematical statistics from Southern Methodist University in Dallas and an MBA from the University of Delaware in Newark. Henderson is a senior member of ASQ.

Reasonableness limits
A nal set of limits that often makes sense for a manufacturing enterprise is a set of reasonableness limits, or a set of limits to identify iers or outliersresults that would be virtually impossible to observe without some obvious error in acquisition and measurement having transpired. Generally, results outside these limits would trigger at least a remeasurement and often an investigation of the source of the highly unusual result.

Value of each limit

Control, disposition and reasonableness limits are more directly related to daily decisions made within a manufacturing operation than are product specication limits: Process control limits drive decisions on process operationwhether to adjust or shut down to investigate issues. Product disposition limits drive decisions on product management and owscrap, rework or further pro-

May 2011 QP 41