This action might not be possible to undo. Are you sure you want to continue?
Vol. 16, No. 1, January-February 2007, pp. 77–92 issn 1059-1478 07 1601 077$1.25
© 2007 Production and Operations Management Society
A Diagnostic Tree for Improving Production Line Performance
Wallace J. Hopp • Seyed M. R. Iravani • Biying Shou
Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208, USA Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208, USA 4R Systems, 1400 Liberty Ridge Drive, Suite 102, Wayne, Pennsylvania 19087, USA email@example.com • firstname.lastname@example.org • email@example.com
mproving performance of production systems is a critical but often unstructured activity. To help managers convert ad hoc or trial & error improvement efforts into efﬁcient and systematic reviews, we develop a diagnostic tree which decomposes a performance improvement objective into successively more concrete sub-objectives and ﬁnally into potential improvement strategies. Based on principles from the Operations Management literature, this tree is structured to enable a non-specialist to better understand the links between corrective actions and performance. It also provides an important foundation for a principles-based knowledge management system that couples the decision tree with a search engine for locating relevant documents within an intranet. Key words: knowledge management; factory physics; diagnostic tree Submissions and Acceptance: Received May 2005; revision received February 2006; accepted April 2006.
From the advent of the factory system during the industrial revolution, manufacturing ﬁrms have been striving to improve critical business processes. To achieve distinction in the marketplace, manufacturers have sought new ways to reduce cost, improve quality, speed response time, and enhance ﬂexibility. In recent years, as product life cycles have become shorter and global competition has intensiﬁed, the importance of continual improvement has become more critical than ever. There exists a vast operations management (OM) literature that provides many insights into manufacturing system behavior as well as methods for improving system performance. Unfortunately, evidence suggests that the industry practitioners have made limited use of these results. For example, in a review of the history of the Institute for Operations Research and the Management Sciences (INFORMS), Horner (2002) commented:
“It didn’t help that traditional management science courses started losing ground in business schools. It turns out that MBAs are more interested in making money than making
mathematical models. Go ﬁgure. The net result: many business schools responded to their customers’ (i.e., students) demands by reducing or eliminating the once-required management science component of their curriculums. That, in turn, further dampened the proﬁle of OR/MS in the corporate world and no doubt led some analysts to reconsider their professional afﬁliation.”
In a similar vein, in a discussion of where Operations Research should go, Dunnigan (2004) said:
“Because OR requires you to think clearly and methodically, there aren’t many practitioners. Despite strenuous efforts, only about ﬁve percent of US army ofﬁcers are familiar with OR techniques (based on an estimate I did with Army OR instructors at Ft. Lee.) And the army has cut back on the training of ofﬁcers in operations research techniques.”
We believe there are two major reasons for this gap between theory and practice. First, the OM ﬁeld covers a wide range of topics, so practitioners may easily get lost when seeking relevant results for their speciﬁc system in the vast literature. This is becoming even more challenging as the OM literature continues to grow. Second, much of the research literature involves
2. it is difﬁcult for practitioners to apply results generated from simpliﬁed OM models directly. we take a ﬁrst step toward developing such a framework by creating a diagnostic tree for evaluating and improving production lines. Literature Review The process of diagnosing problems and searching for improvement opportunities can be considered as a chain of decision-making processes. For this. There are two main categories of information technologies that can help decision making in unstructured problem contexts: expert systems and learning mechanisms. The tree decomposes this objective into subordinate objectives (nodes) which contribute to the parent objective. Examples of expert systems include knowledgebased systems and fuzzy logic. Buzacott and Shanthikumar (1993) noted: “(To have managers believe in models) often requires that the managers have detailed understanding of the techniques of model formulation and solution. The decomposition of objectives at the top levels of the tree is based on generic concepts from the OM literature. complex problems for which there are no cut-and-dried solutions. an inference engine. which makes it difﬁcult for practitioners to implement ideas directly. potential improvement strategies (remedies) emerge naturally. and analyzing possible courses of action. Decisions where some but not all of the phases are structured are called semi-structured. Simon (1977) de- scribed such a decision making process as a threephase process consisting of: • Intelligence: searching for conditions that call for decisions. Appropriately used. as well as developing a broader and deeper understanding of the underlying behavior of their systems. The origin (root node) of this tree corresponds to the objective to improve performance. • Choice: selecting a course of action from those available. pp. at the lower levels of the tree. we have constructed a tree-building tool that makes it easy for experts to improve and update the tree over time (see Birnbaum et al. From the perspective of knowledge management. However. Iravani. Examples include neural networks. In this paper. a prescribed solution can be developed through the use of quantitative formulas or models. 2005 for an overview). Learning mechanism develop rules and/or relationships between data via extensive data training. • Design: inventing. we provide background information on how each sub-objective contributes to the parent-objective and how to determine which sub-objective is relevant to the current situation. we must ﬁnd a way to summarize a broad cross-section of principles and insights from the literature and display them in an intuitive and easy-to-use framework. The expert knowledge may consist of a combination of a theoretical understanding of the problem and a collection of heuristic problemsolving rules that shown by experience to be effective in the domain. Since diagnosing production performance issues and searching for improvement opportunities involves wide ranges of factors. Because of the tractability of mathematical models. As such. Some of these subordinate objectives are then decomposed further. This enables organizations to start out with a basic science-driven tree and evolve a larger practice-oriented tree. This is still rare in manufacturing. Often. we expect them to be relatively ﬁxed. 77–92. An expert system contains three basic elements: a knowledge base. and data mining. decision tree learning. our approach provides a mechanism for codifying tacit knowledge in terms of basic principles (Ferdows 2006). where objectives become more speciﬁc to the environment. An expert system is a computer program that represents and reasons with knowledge of a human specialist with an objective to solving problems or giving advice (Jackson 1990). The expert’s knowledge is contained in . This process makes the knowledge of the OM literature and the organization’s internal experts available to problem solvers with less experience. Expert systems acquire knowledge from human experts and code it into computer programs to aid those with less expertise in solving similar problems. Structured processes involve problems for which standard solutions exist. practitioner improvement efforts have mostly been unstructured and have relied on human intuition as the basis for decision-making. © 2007 Production and Operations Management Society considerable mathematical complexity. As a result. procedures for obtaining the best solution or a satisfying solution are well established.” In order to make the work of the academic OM ﬁeld more available to practitioners. To enable the user to navigate through the tree. In structured problems. Modeling involves the transformation of the realworld problem into an appropriate prototype (usually mathematical) structure. our diagnostic tree can assist manufacturing managers in ﬁnding effective improvement strategies. Decision-making processes fall along a continuum that ranges from highly structured to highly unstructured (Turban and Aronson 1998). and a controller. the tree needs to be more adaptable.78 Hopp. most models in the OM literature are focused on a small part of the real system and omit much detail to allow them to be formulated precisely (Buzacott and Shanthikumar 1993). Unstructured processes are those for which none of the three phases is structured. They are fuzzy. As the decomposition goes on and the sub-objectives become more concrete. developing. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1).
is often considered “the bottleneck” of expert system application. there is not yet a generally accepted incentive system for promoting expert participation. and hence. Another key environmental requirement for making effective use of an expert system is the ability to apply a relatively constant set of rules over time. Expert systems have also seen a number of applications in manufacturing and service management. rules based on human physiology apply to many patients and do not change rapidly. 77–92. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). in medical applications. the knowledge expert needs to learn the speciﬁc domain at some level of technical detail. 1998. Expert systems have been successfully applied to a diverse range of domains. For example. there is no chance that the trained system will suggest checking buffer size as a possible reason for poor throughput. (1998) and Qiu et al. may be too busy to take part in the knowledge engineering process. and internal medicine.Hopp. The premise of our research is that there exists a body of science that describes the underlying principles governing the behavior of a broad range of manufacturing systems. and a domain expert. Knowledge acquisition is usually accomplished by a series of lengthy and intensive interviews between a knowledge engineer. quality and safety. considerable literature has been devoted to discussing incentives for encouraging people to share their knowledge (e. and discovering hidden relationships in data. structural analysis of complex objects (such as chemical compounds). In contrast to the expert system approach. we extract principles and insights from the . Typical tasks for expert systems involve the interpretation of data (such as sonar signals). Examples of other applications can be found in Khoshnevis et al. such as scheduling and quality control. and pricing. There are several possible reasons for this: (i) In order to efﬁciently communicate with the domain expert. but the effectiveness depends on the range of decisions to which these rules can be applied. However. applications like engine diagnostics are similar to medical settings. pp. (2001) present applications for engine diagnosis. can also be subjected to rule-based control. © 2007 Production and Operations Management Society 79 the knowledge base. In fact. diagnosis of faults or diseases.g. Von Krogh et al. Other applications. learning mechanisms discover rules and/or relationships between data via extensive data training. and are therefore usually hard to apply to other environments. and problem classiﬁcation (Anderson 1995). optical character recognition. and planning of sequences of actions (Jackson 1990). the applicability of the generated results are heavily dependent on the speciﬁc domain/context. (iv) Experts may fear that the system will threaten their jobs. Yi et al. Iravani. such as organic chemistry. data validation. but it does not make sense to expend the effort needed to compile such a big set of rules if they will become obsolete after only a few decisions. an expert system can be highly useful. (ii) The facts and principles underlying many domains of interest cannot be clearly articulated in a format that is easily coded into a program. and the controller manages the system by applying the inference engine functionality to the appropriate knowledge. the categories or attributes to which these example cases are assigned must be established beforehand. if the system designer is not aware of the potential impact of buffer size and hence does not provide information in the training cases to examine it. Compared with the expert system methodology. By appealing to this manufacturing science.. In manufacturing. 2000). artiﬁcial neural networks mimic the processes found in biological neurons to predict and learn from a given data set. (1999) and Lin et al. usually hundreds or even thousands of cases. learning mechanisms have the advantage of being capable of learning. In order to diagnose problems in manufacturing process and ﬁnd potential improvement strategies. which is deﬁned as “the transfer and transformation of potential problem-solving expertise from some knowledge source to a program” (Buchanan and Shortliffe 1984). For example. neural networks are used in applications such as pattern recognition. For example. Today. Maus and Keyes (1991) review applications of expert system in manufacturing ranging from scheduling and forecasting. adapting. The productivity of this process is typically low. For example. who is normally a computer specialist. (1996). and error recovery pose further challenges during the development and evolution of a learning mechanism. process control and planning. Drucker et al. their development often requires substantial amount of training data. The inference engine contains the rules of logic and analysis that are applied to information in the knowledge base. To do this. this requires a thorough understanding about the relationships between a wide range of factors. we can save a great deal of time and expenditure by reducing the tedious interviewing process (in the expert system approach) or the long training procedure (in the learning mechanism approach). (iii) Experts are usually in demand. Finally. Knowledge acquisition. mineral exploration. Beyond the difﬁculty of collecting large sets of training examples that contain complete descriptions of variables that could contribute to a poor performance. resource allocation. Moreover. Perez and Koh (1993) describe the design and implementation of expert systems for parametric test analysis for semiconductor manufacturing. diagnostics. to design. Thousands of rules are needed to provide speciﬁc solutions to the problems. hence. since training data are collected from a speciﬁc environment. outcome prediction. However.
this tree is appropriate for manufacturing systems organized into lines. 2005) or focused communication with a human expert. .g.e. which has been shown to be effective in eliciting inﬂuences. project-oriented “bay build” systems and service systems involving collaborative work). Domain Speciﬁc Rules existing literature (see e. the effectiveness of our approach. Moreover. those in the dotted box in Figure 1). beliefs and preferences in a natural and comfortable manner. An inﬂuence diagram begins with an objective node and then expresses achievement of that objective as a function of the achievement of several lowerlevel objectives. Iravani. a modiﬁed tree is needed. We give a high level schematic of the resulting diagnostic tree in Figure 1. While a theory based tree can provide reasonably comprehensive guidance at a generic level.. like that of inﬂuence and ﬁshbone diagrams. the diagnostic tree will remain relatively robust over time and environment setting. they would leave to the user to identify the cause-and-effect relationships. The process continues iteratively until one identiﬁes actionable drivers of the overall objective. While a generic inﬂuence or ﬁshbone diagram could be applied to the class of problems we consider. The primary difference of our approach from these previous methods is that it has been designed explicitly to address Operations Management issues. is premised on the belief that scientiﬁc decomposition of a problem (objective) and thorough investigation of alternatives outperforms ad-hoc solutions aimed at symptoms rather than underlying causes. we leave the problem solving to a search engine that can identify relevant documents (see e. • Fishbone diagrams: are also known as “Cause and Effect diagrams” or “Ishikawa” diagrams after their creator Kaoru Ishikawa (Isikawa 1985). We appeal to the basic principles to generate the generic-rules portion of the tree. This science-based approach enables our diagnostic tree to provide comprehensive coverage of major alternatives relevant to a given setting. summarize a wide range of basic relationships. intuitive information. At the point where objectives decompose into domain speciﬁc rules (such as adjusting the speciﬁc gravity of the copper plating solution to reduce incidence of “dish down” defects). Because the above literature focuses on production lines where parts ﬂow between operations separated by intermediate inventory buffers. and in providing a computational substrate for efﬁcient inference (Horvitz 2005). We also share the graphical approach of these methods.. Buzacott and Shanthikumar 1993. managers can substantially narrow down their search and structure their thinking about a solution to their particular problem. Suri 1998. 77–92.g. For other environments (e. by decomposing higherlevel performance metrics into generic classes of improvement objectives (i.g. while remaining rigorous methodologically..g. Birnbaum et al. Keeney 1992. There are two related approaches for reporting and analyzing decision problems that also make use of an iterative decomposition process to generate alternatives: • Inﬂuence diagrams: were introduced in the late 1970’s by Ronald Howard and James Matheson and have since become widely used in decision analysis (see e.80 Figure 1 Hopp. Askin and Standridge 1993. Horvitz 2005). Our approach shares the general objective of inﬂuence and ﬁshbone diagrams to provide clear. By avoiding codiﬁcation of overly detailed domain speciﬁc rules. it cannot enumerate environmentally speciﬁc suggestions. Anupindi et al. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). By using the principles of factory physics. pp. Ultimately... Hopp and Spearman 2000). and distill these into a hierarchical tree that links performance metrics to improvement policies. 1999. © 2007 Production and Operations Management Society Generic Rules vs. They are widely used in Total Quality Management (TQM) to systematically enumerate the various causes of a speciﬁc problem or effect.
which shows how the current node/objective may be decomposed further into more concrete sub-objectives. This information helps users keep track of the diagnosis process and also easily return to previous nodes and continue exploring the tree from them. When the decision process reaches the more detailed levels of the lower portions of the tree. © 2007 Production and Operations Management Society 81 we are able to provide a substantial amount of structure to the diagnostic tree. At any objective node in the tree. The text following “A note on parallel tasking” further clariﬁes why “parallel tasking” is beneﬁcial by providing an example. improving quality. as is typical of the pages for most nodes. we present only a small portion of our D-Tree in Figure 2. and what subordinate improvement strategies may contribute to the current objective. 3. and to incorporate artiﬁcial intelligence guided searches of the Internet and/or an Intranet to provide the user with additional guidance (see Birnbaum et al. Due to space constraints. Iravani. To our knowledge. We refer the reader to http://km. (III) The navigation section. it can help propose effective solutions.Hopp.edu/tree/ viewnode. this tool can provide guidelines for managers.mccormick. For example. Suppose the user has decided that reducing cycle time is crucial to remain competitive. reducing cycle time.northwestern. the ability to incorporate a body of knowledge at the generic level. and clicks “Reduce cycle time”. The page in Figure 3. which provides the links to the immediate subordinate nodes of the current node. and (2) “Enhance parallel tasking”. the diagnosis node explains how “reduce cycle time” may be achieved via two avenues: (1) “Reduce excessive station cycle time”. Section 3 describes the elements of the diagnostic tree. In this example. diagnosis and decomposition are then performed upon these nodes. let us consider an example. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). such as which important concepts and mathematical formulas are relevant to the current context. as well as the diagnosis path leading to the current node. it can serve as the foundation of a system that evolves over time. decomposes this objective into successively more concrete subordinate objectives. the user considers the situation of the manufacturing line as well as the information provided by the tree to decide which subordinate node(s) may be relevant. Finally. It further suggests that when large delays are common the former is likely to be an attractive strategy. armed with learning mechanism. (II) The content of the diagnosis node associated with the current node. it shows that the current node is “reduce cycle time” and it was reached directly from the root node “Production system performance improvement”. On its own. • Diagnosis nodes explain the related science. In this case. 77–92. but when jobs are processed in large batches or there is opportunity to divide work so that it can be processed simultaneously at different stations the latter is likely to be attractive. such as “reduce cycle time” or “improve throughput”. factors that may make “Increasing bottleneck capacity” attractive include “high bottleneck station utilization” and “additional capacity is easy and cheap to obtain”. what data analysis should be performed. The ﬁrst level of the tree consists of the primary performance objectives of a production system: improving throughput. conclusions and discussions are presented in Section 5. improving customer service. where users may input their comments (such as thoughts on why they think certain subordinate objective may be effective for their pro- 3. the user must make use of local environmental knowledge to identify relationships. The user selects the objective of interest and then. New rounds of . The diagnosis nodes also provide a list of factors stating when each subordinate improvement strategy may be attractive. the diagnostic tree we propose in this paper is the ﬁrst attempt to aggregate the science of production lines into such a concise and application-oriented framework. Exploring the Diagnostic Tree To illustrate how this D-Tree translates OM principles into simple decision rules. links to “Reduce station cycle time” and “Enhance parallel tasking” are listed. Section 4 illustrates the value of the tree by applying it to a real-world case study. (IV) Comments box. However. adapts to a speciﬁc domain. In this case. contains four major components: (I) The name of the current node. So our method devolves to a version of a inﬂuence/ﬁshbone diagram at those levels. This simpliﬁes the users task by encoding the established body of knowledge into the generic rules in the high level portions of the tree.1. with the aid of the D-Tree.php for a complete web-based representation of the tree. Diagnostic Tree The primary elements of our Diagnostic Tree (D-Tree) are objective nodes and diagnosis nodes: • Objective nodes represent the performance objectives of concern to managers.  for a description of this hybrid decision tree/search engine approach) makes our approach particularly well-suited to diagnosing and improving production systems. The remainder of the paper is organized as follows. and evaluates the effectiveness of speciﬁc solutions. combined with more detailed knowledge of a domain expert. This leads to the web page shown in Figure 3. and reducing cost. pp. The process continues until it reaches a node(s) that is concrete enough to serve as a guideline for practice.
which leads to the web page as shown in Figure 4. A tree structure nicely reﬂects the decomposable nature of the underlying factory physics knowledge base. and component III provides the links to them. component IV provides space for user comments. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). which lists the comments the user saved while examining the previous node “Reduce cycle time”. If we click it now. and yield.82 Figure 2 Excerpt of Diagnostic Tree Hopp. Example I: A Variability Benchmark. For instance. (We will demonstrate how this decision might be reached in a case study in the next section. pp. we will be led to the web page shown in Figure 5. Iravani. at the same time. the formula for throughput calculation indicates that one may increase throughput via improving three factors: utilization. the user observes that large delays on stations are frequently observed in her production line. so reducing station cycle time seems like a promising direction for line cycle time reduction. 2. 77–92. In this case. 3. Below we provide two examples of such benchmarks that are part of our diagnostic tree. After considering this information. Benchmarks Some diagnosis nodes may include benchmarks from industry standards.) Suppose also that she observed excessive blocking at . © 2007 Production and Operations Management Society duction system) and save them. “Comment Summary”. There are two major advantages of having a diagnostic tree to represent and organize knowledge: 1. “Why is this happening? What is causing this?” This helps operations managers structure their reasonings to ﬁnd the most effective improvement strategies. It starts with the effect and the major groups of causes and then asks. Compared to mathematical equations. the diagnostic tree highlights the chain of causes. Note that now component I shows the diagnosis path to the current “Reduce station cycle time” node. jobs are currently processed in small batches. the tree is much more accessible to practitioners. Suppose a user has decided that improving bottleneck utilization is attractive. capacity. but diagnosis nodes can also make use of internal benchmarks derived from mathematical models.2. the user clicks the link “Reduce station cycle time” in the navigation section. component II shows how reducing station cycle time may be further decomposed into subordinate objectives. that provides easy access to all previously saved comments. Like a ﬁshbone diagram. Note that there is a link on the top right of the window. so enhancing parallel tasking is not likely to be helpful. Again.
the diagnosis node provides the following benchmark for variability derived from the M/M/1/b queueing model: Example II: A Repair Time Benchmark. Figure 4 Sample Web Page II: Reduce Station Cycle Time . 4. it provides the core body of knowledge needed to systematically address a complex improvement problem like that described here. Note that in the previous example. possibly in combination. our approach will be useful even if the diagnostic tree is not exhaustively comprehensive. a problem-oriented tree could induce a user to stop with the ﬁrst cause identiﬁed and thereby miss productive options. In one of their plants they made printed circuit boards (PCB’s). including “Reduce queueing effects”. However.. Inc. the coefﬁcient of variability exceeds one) indicates a problem. Furthermore. © 2007 Production and Operations Management Society 83 Figure 3 Sample Web Page I: Reduce Cycle Time several improvement alternatives may be appropriate. we consider the next level below the objective node of “Reduce process variability” as shown in Figure 7. HAL. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). The diagnosis nodes provides information useful for determining when each of these options are likely to be attractive.000 square feet of manufacturing space. Inc. because the objective is to generate attractive alternatives. Case Study We now illustrate the application of our diagnostic tree to a real-life case study. pp. “Increase downstream capacity”. The analysis of this case was carried out prior to the development of the tree. Our solution-oriented tree encourages the user to think in terms of generating a list of alternatives. which were used by other plants in the company in a variety of computer products.Hopp. is a major manufacturer of computers and computer components. the tree encapsulates the major insights used to solve the case.e.5 the bottleneck station. 77–92. note that we have framed the process as a search for improvement options instead of a search for “the” problem. We do this because poor performance may not be the result of a single problem. Hence. the diagnosis process actually involves a combination of an external benchmark with an internal (model based) one. Finally. In contrast. as we show. Iravani. Hence. As such. The basic process ran three shifts per day (19. there is an implicit assumption that variability in process times such that the standard deviation exceeds the mean (i. HAL. The plant under consideration represented an $80 million investment and had approximately 450. rather than to identify a single problem. The diagnosis node “Reduce bottleneck blocking” (Figure 6) articulates the options that may contribute to the objective of reducing bottleneck blocking. To further illustrate how models can be combined with external benchmarks to provide useful diagnostic tests. For example.
External Circuitize: using the same photographic Figure 7 Diagnosis Node—Reduce Process Variability Figure 6 Diagnosis Node—Reduce Bottleneck Blocking . from which all of the ﬁnished boards are made. giving the cores “personality” (i. Internal Circuitize: through a photographic exposing and subsequent etching process. 6. 77–92. 4.84 Figure 5 Hopp. Iravani. 2. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). 3. Treater Process: woven ﬁberglass cloth is impregnated with epoxy to make “prepreg. lunches. pp.” the insulator used in multi-layer printed circuit boards. © 2007 Production and Operations Management Society Sample Web Page III: Comment Summary hours/day taking into account breaks. Lamination-Core: layers of copper and prepreg are pressed together to form cores (blank panels from which boards are cut). 5. Lamination-Composites: circuitized panels are pressed together into multi-layer panels. which are repaired if not too severe. circuitry is produced in the copper layers of the blanks.. Machining: the cores are trimmed to size. and shift changes) and involved the following major steps: 1. Optical Test and Repair-Internal: the circuitry of the cores is scanned optically for defects. 7.e. There are 8 different core blanks. a unique product character).
Panels are loaded by an operator. ﬁve days a week. 3. since the different products had different manufacturing complexities. on the expose machines. the plant manager felt that an average throughput of 3000 panels per day was a reasonable goal. Because actual throughput was consistently falling short of this level. one board at a time. circuitry is produced in the copper layers on the outside of the laminated panels to provide additional layers of circuitry. Depending on the size of the board. A worker (generally someone from Manufacturing Inspect or the operator who loads the coater) takes Figure 8 Procoat Process . which had been averaging only around 1200 panels per day for the past several months. thereby connecting the circuits on different planes. The cost of vendoring plus shipping was around $15 per board. multiple boards are manufactured on the same panel and are cut into individual boards at the sizing step.2. HAL had engaged a vendor to perform the coating operation for the panels that they were not able to coat in-house. Despite these complexities. Iravani. Therefore. which are repaired if not too severe. he began calling a daily meeting of his line managers to discuss the details of why the target for that day was or was not met. © 2007 Production and Operations Management Society 85 exposing and etching process used to expose cores. Moreover. Drilling: holes are drilled in the panels to connect circuitry on different planes. Each expose machine requires two operators. In most cases. 12. 4. An operator loads the panels into a loader. pp. some products had many holes. Optical Test and Repair-External: the circuitry of the external layers is scanned optically for defects. Coat 2. 11. End-of-Line-Test: an electrical test of each board’s functionality is performed. which hardens the plastic coating and are accumulated on the other end by another unloader..e. By using a special piece of ﬁlm (called a diazo) these machines expose the panels to UV light. For instance. one job at a time. 13. the process that limits production) depended on the product mix being run. The purpose is to remove the plastic coating from areas of the board that must not have it (e. this vendoring introduced an additional two-day delay because of shipping (up and back). which. coats the other side. 6. because the vendor was remotely located. Consequently. pads for mounting power supplies). which feeds them into the coater. Problem Statement The stated goal of the plant was to run 3000 panels per day.1. 2. the capacity of the plant and possibly even the identity of the “bottleneck” (i. which cleans the panels. which makes the exposed plastic resistant to dissolving in the subsequent developing process. which deposits a liquid plastic coating on one side of the board. 5.Hopp. they pass down a short conveyor. there could be as few as two boards on a panel. 8. 4. which deposits copper inside the holes. the plant manager was very anxious to ﬁnd ways to increase the throughput of the Procoat process. Procoat: a protective plastic coating is applied to the panels. However. a signiﬁcant cost. The panels are photographically exposed. The coater is a conveyor that consists of three steps: Clean. 10. onto an automatic loader.. He/she then replaces the board on the conveyor feeding the Bake oven. Procoat Process The process of Procoat involves the following steps (see Figure 8): 1. after ﬂipping the panels over. As a result. this goal could only be a rough one. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). 77–92. Coat 1. A D&I (for Develop and Inspect) inspector must take each board off the conveyor and check it for defects and record the results on an adjacent table. which feeds them into the Develop bath. 9. Copper Plate: the panels are run through a copper plating bath.g. The panels travel through the Bake oven. Panels are accumulated by an automatic unloader and an operator (who may be an expose machine operator or another clean room worker) places them in carts in the clean room that contains the expose machines. 4. with the plant running three shifts per day. Panels arrive at Procoat from Copper Plate in carts containing about 60 panels each (the number of panels per cart (job) can vary slightly due to yield loss at upstream operations). or as many as twenty. causing them to move through Drilling very slowly. One of the processes that was identiﬁed during these meetings as a particular problem spot was Procoat. 7. Sizing: boards are cut from the panels to their ﬁnal size. while others had fewer holes and thus required less time at Drilling. As the panels emerge from Develop.
95 0. management went through (with a little help from two consultants) the analysis outlined below. which is the capacity of the station. © 2007 Production and Operations Management Society Machine name Clean 1 Coat 1 Coat 2 Expose Develop Inspect Bake MI Touchup Conveyor trip time (min) 15 15 15 — 2.97 0. and “Time”.33 161 9 Std dev process time (min) 0 0 0 67 0 0.5 0 64 0 Hopp. At the end of Bake.00 0.86 Table 1 Procoat Data Process of load time (min) 0. inspectors check each board with a magnifying glass for defects in the coating. “MTTR” is the Mean Time to Repair. Diagnosis The diagnosis node of “Improve throughput” is shown in Figure 9.0 the carts of panels unloaded from Bake oven and moves them to another room containing Manufacturing Inspect (MI).33 0. in addition to being very expensive. “Conveyor Trip Time” is the average time a job spends in a conveyor-type operation.95 0.5 121. Although the actual process was not quite as smooth as traversing the tree in Figure 1. the inverse of the process or load time) by availability and setups to compute the “Rate”. The following parameters are basic inputs for each stage of the process: “Process or load time” is the average process time of a one-at-a-time operation and the time required to load a job on a conveyor-type operation.3.00 1. 8. A simple capacity analysis showed that Expose was the bottleneck (see Table 1). Given this.. A worker (currently the inspector between Develop and Bake) applies additional liquid coating to the defective panels and feeds them into the Bake oven for re-baking. “Number of Machines” is the number of (identical) machines in the station.9 22. these panels are rejoined with the others in their job and are sent out of Procoat to Sizing.99 1. installing more machines would require expanding the clean room in which the Expose pro- cess was performed.0 161.5 36. “Std Dev Process Time” is the standard deviation of the process or load time. This would add signiﬁcantly to the cost and would result in substantial production loss during construction. “MTTF” is the Mean Time to Failure. In hopes of ﬁnding an alternate (and cheaper) way to improve Procoat throughput.5 0.99 1. which is the average time spent in the station.33 0. Iravani. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1).0 9. 4. “Avail” is the availability which equals MTTF/(MTTF MTTR). With these.5 36. the responsible engineers were in favor of installing more Expose machines. “Setup Time” is the average setup time. we will present the actual conclusions using the tree as an organizing framework. Unfortunately. Defective panels are marked and jobs containing defective panels are sent to the Touchup table.00 Setup time 0 0 0 15 0 0 0 0 0 Rate (p/day) 3377 3377 3377 2879 3510 4680 3510 3488 7800 Time (min) 36.67 — 100 — — Number of machines 1 1 1 5 1 2 1 8 1 MTTF 80 80 80 300 300 — 300 — — MTTR 4 4 4 10 3 — 3 — — Avail 0.33 0.7 0.5 121.e. Capacity and other data for the Procoat line are given in Table 1. we adjust the stated rate (i.33 103 0.95 0. 77–92. Had this tree Figure 9 Diagnosis Node—Improve Throughput . Jobs with no defects are sent out of Procoat to Sizing. pp. There.
• Add operators: At the time of this analysis. Manufacturing Inspect) were not highly utilized. • Transferring capacity: Operators at some other stations (e. the option “Increase post-bottleneck yield” can be eliminated.) • Train operators: Because the fastest Expose operators achieved signiﬁcantly higher output rates than the slowest operators. installing another Expose machine would require expanding the clean room. we now explore how to “increase bottleneck utilization”. this is the ﬁrst place to examine. “increasing bottleneck capacity” is clearly an important option to consider. However. Considering each of these in order led to the following conclusions: • Reduce bottleneck blocking: Expose was occasionally blocked by downstream processes. management was able to systematically identify potential attractive opportunities (as illustrated in Figure 11): • Increase shift time: Because break plus lunch time accounted for about 20% of shift time (see Table 2). the tree offers several generic options as shown in Figure 10. each of which required two operators to run. (2) purchasing new machines or upgrading to faster models would be very expensive. The tree enumerates the factors that reduce bottleneck utilization and offers the generic options shown in Figure 12. Iravani. is to consider the three subordinate objectives: • Increase bottleneck capacity • Increase bottleneck utilization • Increase post-bottleneck yield After collecting and analyzing yield data for the Procoat line. However. there were only six operators (four on third shift) on staff. which would add signiﬁcantly to the cost and would result in substantial lost production during construction. “improving bottleneck utilization” may also offer opportunities. the . it would almost certainly have sped things along. The next step. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). Hence. the stafﬁng shortages Figure 10 Diagnosis Node—Increase Bottleneck Capacity were the result of some exogenous events that had occurred over the two months prior to the analysis. installing more machines was unattractive because: (1) the current Expose machines were not fully utilized (utilization of Expose process 66%). 77–92.g. since utilization of Expose is only about 66%. © 2007 Production and Operations Management Society 87 been available when the analysis was done. Having examined options for increasing bottleneck capacity. By considering each of the above. • Upgrade equipment: The only known option for improving Expose machines was to increase the durability of the negatives used in the expose process so that the number of setups can be reduced. Likewise. By logically enumerating the mechanisms by which bottleneck capacity can be increased.Hopp. management knew this. it became clear that yield loss was not a problem at Expose or the operations downstream from it. since the capacity of the Expose process is smaller than the target throughput of 3000 panels per day. as the diagnosis node “Improve throughput” suggests. Because Expose is the bottleneck. • Reduce rework: The amount of rework at Expose was very small. • Install more machines: As we noted above. So an obvious step for increasing throughput would be to fully staff the Expose machines. it appeared that identifying and disseminating best practices through training could be an attractive avenue. there was some opportunity for increasing shift time by moving workers from one operation to cover the bottleneck during breaks. In addition. pp. We discuss this option in more detail later. (Of course. This suggested that some kind of work sharing could be used to shift capacity from these lowly utilized stations to the bottleneck.. there were ﬁve Expose machines. so there was not much room for improvement via reducing rework. However.
Making a second D&I operator available would easily resolve this problem. But this would allow the buffer to become nearly empty. were lunches/breaks. The existing policy was to Expose Work Stoppages Duration (min) 20 40 10 90 Quantity 2/shift 1/shift 1/shift 1/week • shut down the Coater line for three hours whenever the buffer in front of Expose became full.e. it is not possible to build up a large inventory buffer with which to protect Expose from upstream disruptions. he/ she must periodically request a halt in the loading of the Develop process in order to catch up. Therefore. identifying the causes of these disruptions is important to reducing the frequency of starving. pp. it lacked the other key feature of pull—a mechanism for efﬁciently communicating WIP voids upstream. Iravani. other than blocking and starving. Reduce bottleneck starving: Expose was frequently starved by the Coater process. to enable the line to achieve the efﬁciency of pull (i. However. which meant that Expose was particularly vulnerable to any disruption in ﬂow.88 Figure 11 Hopp. Of course. © 2007 Production and Operations Management Society Diagnosis Process—First Steps • • capacity analysis in Table 1 reveals that this is not due to the capacities of the downstream processes themselves. but how lunches/breaks were Figure 12 Diagnosis Node—Increase Bottleneck Utilization Table 2 Work stoppage Break Lunch Shift change Meeting . 77–92. the system already had one key feature of a pull system. Because there is only space for ﬁve carts of WIP in the Expose clean room (as shown in Table 3). the ability to produce greater throughput for a given WIP level). as the downstream capacities are greater than the targeted throughput. Therefore. Implement pull: Because of the ﬁnite buffers in the line. Eliminate unnecessary stoppages during shifts: The two main reasons Expose would stop producing during a shift. Because a single operator cannot keep up with the inspection and paperwork duties when Expose is running at full capacity. these were contractually required. Rather. namely that it implements a WIP cap that prevents WIP from growing without bound. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). a tighter connection is needed between the front of the line and the buffer in front of Expose.. it is a consequence of inadequate stafﬁng at the D&I Inspect station.
So the tree must be pursued further to ﬁnd out why. so this is not a source of opportunity. So improvements might be possible by identifying and adopting best practices. downtime was not a signiﬁcant loss of production time at the bottleneck. To keep our presentation concise. The diagnosis node. pp. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). while rework has been discarded as a • Figure 15 Diagnosis Node—Reduce Process Variability . the availability of Expose machines was 97%. To follow up on the options suggested by the above. These setups were technically necessary because the negatives used to expose the appropriate patterns on the panels wore out after two jobs. setups accounted for 14. • Reduce equipment downtime: According to Table 1. Reducing setup time loss has already been determined to be attractive. Iravani. but the time they took was subject to the efﬁciency of the operators doing the setup. After gathering additional data it was found that the process variability was high at both the Expose and Coater processes. Following the tree. this is the alternative that needs exploration. according to Table 1. “improve upstream process capacity” is not a promising option. so there was little opportunity for improvements in this dimension.Hopp. “reduce process variability” reads as shown in Figure 15. Considering each of these in order yields: • Reduce upstream yield loss: Data showed that there is no signiﬁcant yield loss upstream from Expose. The tree expands the node for “Reduce queueing effects” and articulates the avenues for achieving this objective as shown in Figure 14. • Reduce setup time loss: According to Table 1. • Increase upstream process capacity: Since the capacity of the Coater process is already larger than the target throughput and the utilization of the Coater machines is quite low (about 34%). we will probe the options to “Reduce bottleneck starving” and “Reduce setup time loss” in detail and will only give overviews of the outcomes for the other options. the diagnosis node “Reduce bottleneck starving” is shown in Figure 13. 77–92.6% of Expose process times. we move to the next level of the tree. Figure 13 Diagnosis Node—Reduce Bottleneck Starving Reduce queueing effects: Given that the above two are not attractive. © 2007 Production and Operations Management Society 89 Table 3 Inventory Buffers Capacity 5 4 40 15 10 Jobs or panels Jobs Jobs panels Jobs Jobs Figure 14 Diagnosis Node—Reduce Queueing Effects Buffer location Before Expose After Expose At D & I Before MI Before touchup synchronized among the various operators was open to discussion.
33 hours and the availability A 0. let us explore the other option “Reduce setup time loss” as shown in Figure 17. because this risks starvation at Expose.4. so the “Standardize procedures” and “Train operators” options are likely to be attractive. • Improve coordination: As we noted earlier. In this case. the manager decided that the attractive strategies included “Improve equipment”. which signaled available buffer space in the clean room. Automation was also dismissed as overly expensive. we consider the options of “Reduce equipment downtime” and “Improve coordination”: • Reduce equipment downtime: Note that with Coater processing time t0 0. Outcome The overall result of the analysis using the logic of the diagnostic tree was the following list of proposed improvement actions. average repair times of four hours for the machines on the Coater line were not excessively long given their complexity and cleaning requirements. Similarly. Data also showed that waiting time for repair is not an issue since the machines usually receive Figure 17 Diagnosis Node—Reduce Setup Time Loss Figure 16 Diagnosis Node—Reduce Equipment Downtime repair attendance immediately. A much better policy is to launch a new job into the Coater line whenever space becomes available in the clean room buffer. We continue to pursue the tree to the next level to identify options to “Reduce equipment downtime” as shown in Figure 16. They suggested purchasing duplicates of these modules so that they could be quickly swapped out. it is not a good policy. which is much smaller than the average repair time of 4 hours for the Coater machine. © 2007 Production and Operations Management Society non-problem. While the option to “Decrease equipment failures” would certainly help reduce the frequency of Expose starvation. as we discussed previously. But. allowing the failed modules to be repaired after the Coater line had been restarted. Hence.97 (see Table 1). some Expose operators achieve signiﬁcantly better output than others. the maintenance technicians conﬁrmed that a substantial part of the four hour repair time consisted of cleaning and repairing the spray modules once they were removed from the machines. by developing long life negatives that would reduce the number of setups required. as shown in Figure 18. Now that we have examined the option of “Reduce equipment downtime” in depth.90 Hopp. Hire more operators to fully staff Expose machines.15 hours. by having the most effective operators train the rest in best practices. Have operators from Manufacturing Inspect cover Expose during lunch breaks for all three shifts. 77–92. Improve coordination between the Coater and Expose by setting up lights to signal that a job should .75t0/(1 A) A gives 0. reducing repair times for the Coater machine appears to be an attractive avenue to pursue. Moreover. This could be implemented by setting up a light visible by the operator in charge of loading the Coater. pp. Iravani. Of these alternatives. the failure frequency of the Coater line was not excessive compared to other equipment of comparable complexity. management turned to the remaining option to “Externalize repairs. 2. 1. 4. Finally. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). Hence. current practice was to shut down the Coater line for three hours whenever the clean room buffer became full. 3. the internal benchmarking 0. and “Standardize setup procedures”.” The basic idea of externalization of repair times is to do as much of the repair while the line is up and running.
from an average of 1200 per day to an average of 2600 per day. as this example illustrated. By applying our diagnostic tree to a real-world case study. Iravani. which both ensures comprehensive consideration of major alternatives and greatly reduces the need for complex mathematics. This change.7%. Finally. The development of our diagnostic tree is one of the . combined with similar steps at other key processes in the overall line transformed the panel plant from a struggling problem spot to a showcase of efﬁcient manufacturing practices. As such. These action plans were adopted and implemented in the plant. by bringing together results from a broad cross section of the operations management literature. © 2007 Production and Operations Management Society 91 Figure 18 Summary of the Diagnosis Process be started on the Coater whenever there is buffer space available in the clean room. Within three months. 5. 5. throughput of the Procoat line had increased by 116. 4. The outcome was extremely positive. a tree does not automate the entire improvement process. pp. A tree decomposes the problem of improving performance in a manufacturing line into logical steps. So. the tree can suggest externalizing equipment repairs as a possible option for certain situations. Conclusions and Future Work The central thesis of this paper is that a diagnostic tree is an effective way to make the principles of operations management research accessible to practitioners. the analysis is still a very necessary part of the process.Hopp. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). we have illustrated that it is a practical and effective tool for evaluating and improving production line performance. 77–92. For instance. 6. but it cannot tell the decision maker how to accomplish it. Of course. while a tree can help structure a manager or an analyst’s thinking. a tree eliminates the potentially confusing problem of choosing the right model or right result from the literature. a diagnostic tree must stop short of domain-speciﬁc options. Train Expose operators in best practices (identiﬁed from those of the most productive operators) in both setups and processing steps. Stock replacement modules for the Coater machines to facilitate rapid repairs. it has the potential to promote usage of these principles more effectively than the more conven- tional form of problem-speciﬁc mathematical models. Develop long-lasting negatives for the Expose process. In order to be broadly applicable.
P. M. Factory physics. Cataloging expert systems: Optimism and frustrated reality. E. New York. Y. J. W. New Jersey. VerDuin. 2004. Massachusetts. E. OR/MS Today. B. 1998. 2001.. Jackson. C. Englewood Cliffs. New York. Song. U. R. J. R. A. Chu. W. Zemel. Harvard Business School Press. Cavanagh. New York. Chang.. Modeling and analysis of manufacturing systems. Stochastic models of manufacturing systems. Ishikawa. The six sigma way team ﬁeldbook: An implementation guide for process improvement teams. Russell. J. W. Sitarama. 2005. Decision Support Systems 35 487C–515. FLEXPERT: Facility layout expert system using fuzzy linguistic relationship codes. 1993. Standridge. R. 1999. Englewood Cliffs.. Artiﬁcial intelligence: a modern approach. Birnbaum. 1993. Boston. Inﬂuence diagrams. Ichijo. Englewood Cliffs. R. A. 2000. P.. Berkeley. Acknowledgments The authors would like to thank Thomas Tirpak of Motorola for his guidance of this research. More work needs to be done in order for the tree to be more comprehensive. S. J. 1996. D. Pande. Harlow. We have developed a prototype system that does this (see Birnbaum et al. Hopp. 1984. Hopp. M. 1998. Prentice Hall. M. Principles of operations management. T. Dunnigan. to where? emphDSSResources. D. Englewood Cliffs. October 2002. Lin... R. 1990. Productivity Press. What is total quality control?: The Japanese way. Livingston. Brown. Value-focused thinking: A path to creative decisionmaking. S. Applied Intelligence 9 231–243. B. S. S. P. J. The new science of management decision. 2003. Drucker. Wiley Text Books. 1998. A.. Boston. Q. New Jersey. J. University of California. Patterson. Englewood Cliffs.. Transforming the tree introduced in this paper into a full-featured expert system for speciﬁc industries would require adding a mechanism with which to compile domain-speciﬁc information and convert it into additional decision rules for the lower levels of the tree. 5th ed. Van Mieghem. New York. 1991.. INFORMS celebrates 50 years of problems. 2000. 1990. S. B. L. Furthermore. M. Buchanan B. Quick response manufacturing: A companywide approach to reducing lead times. Deshmukh.). New Jersey. P. A knowledge-based decision support system for the management of parts and tools in FMS. Horner. D. 1996. McGraw-Hill/Irwin. Tirpak. Keyes.H. Roy. P. Englewood Cliffs. Chopra. Shanthikumar. Ludden. J. A knowledge-based decision support system for locating a manufacturing facility. Liu.. Massachusetts. An expert system for the design and analysis of composite structures. and Shou: A Diagnostic Tree for Improving Production Line Performance Production and Operations Management 16(1). IIE Transactions 27 789 –799. 2005. MIT Press. Prentice Hall. D. Introduction to expert systems. and as a result. 1993. E. Englewood Cliffs... Keeney. New Jersey. for his help and encouragement. J. D. S. O. in order to give decision makers access to white papers and other data in a ﬁrm’s intranet that are relevant to the situation described by the path through the tree. R. B. Decision Analysis 2 127–143. A fuzzy diagnostic model and its application in automotive engineering diagnosis. 1995. 2000. Transfer of changing production know-how. pp. H. Jungthirapanich. Englewood Cliffs. A. 1995. Enabling knowledge creation: How to unlock the mystery of tacit knowledge and release the power of innovation. W. New Jersey. New York. 2005. Anupindi. Buzacott. MA. Prentice Hall. 1992. 2005). the tree will grow as it is utilized.G. G. Perez. Suri. G. Davidson. (Lu. IIE Transactions 31 303–312. Massachusetts.92 Hopp. C. B. McGraw-Hill/Irwin. Yi. April 4. History in the making. © 2007 Production and Operations Management Society ﬁrst attempts to summarize scientiﬁc results for performance improvement of production lines. Garvin. A. Bell. Agogino. New results can be easily added to the tree. Horvitz. Production and Operations Management 15(1) 1–9. Oxford University Press. Working paper. Lin. Integrating expert systems with a relational database in semiconductor manufacturing. 2nd ed. IIE Transactions 32 573–583. W. S. Decision Analysis 2. Q. 1st ed. Oregon. J. Turban. solutions. Handbook of expert systems in manufacturing. Heizer. M.COM. Khoshnevis. Introduction to artiﬁcial intelligence & expert systems.. K. L. Benjamin. E. 2000. G. Ozbayrak. IIE Transations 28 295–308. Qiu. Neuman. Iravani. Prentice Hall. An integrated process planning system using feature reasoning and space search. Shortliffe. and National Science Foundation for support under Grant DMI-00114598. Proceedings of International Conference on Intelligent User Interfaces. Susan. 1977. A. Knowledge-based systems for manufacturing. N. anecdotes and achievement. Boston.. Prentice Hall. Managing business process ﬂows.. L. T. Cambridge. Sormaz. J. 1998. K. Addison-Wesley. but there remains a great deal to be done in order to realize the full potential of a hybrid tree/search expert system.. K. The operations research revolution rolls on. it makes sense to combine the diagnostic tree approach with search capability. R. Journal of Control Systems and Technology 4 133–138. Askin. New Jersey. Massachusetts. Badiru. T. Portland. Nonaka. Y. I. R. Dorothy. Editorial: Graph-based representations for decision analysis. trans... New Jersey. References Anderson. Beyond growing this tree and developing trees for other production environments. December 14. K. California. Iravani. Boston. 2003. J. IEEE Transactions on Semiconductor Manufacturing 6 199 –206. the Editorin-Chief. January 4. “Rule-Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project”. Task aware information access for diagnosis of manufacturing problems. Intelligent Manufacturing 1(2). New Jersey. Olmstadt. Park. R. England. Aronson... Cambridge. C. New York. R. Addison Wesley Longman. Harvard University Press. Z. . McGraw-Hill. Von Krogh. Prentice Hall. F. C. R.. S.-C. C. 2003. A fusion of bayesian and fuzzy analysis for print faults diagnosis. July 1999. 2001. J. A. Kalyan Singhal. P. J. 77–92.. an interesting future research area is the interface between a diagnostic tree and a human decision maker. 2006. A knowledge acquisition model for selecting coordinate measuring machines using inductive learing. P. Spearman. February 1995. Shou. and E. Journal of Southern Academic and Special Librarianship 01. Matheson. Tie. IIE Transactions 31(7) 597– 616. C.. 1985. L. Arif. J.. J. Cement roller mill control by fuzzy logic controller. F. R. Norvig. 1st ed. Simon. Howard. Department of Mechanical Engineering. Harvard business review on knowledge management. Ferdows. S. S. Prentice-Hall Inc. W. Koh. Maus. S. Wu. R. K. Prentice Hall. Render. Decision support systems and intelligent systems. 1993. An introduction to neural networks. Massachusetts.
This action might not be possible to undo. Are you sure you want to continue?