which is Company C. (More than that, Company A actually nominated Company Cas a potential supplier to Company B.) Company B's engineering division devel- ops a functional specification for the transmission that includes functional char- acteristics, maintenance requirements, interfaces with the rest of the vehicle, and test requirements. Its vendor quality section then verifies that Company C's engineers will be using appropriate processes to ensure cost-effective compli- ance with the specification, and that the transmissions will be tested according to Company B's functional specification to insure compliance to Company B's performance criteria before any transmissions are shipped. 9.3 TECHNIQUES FOR QUALITY ASSURANCE DURING System DEVELOPMENT System developers employ a wide range of techniques to ensure the quality of the project end-item or products. This section discusses a “toolbox” of these techniques. Configuration Management” During the design of a system, vast amounts of data and information are generated for use in the design process and later for manufacturing (or construction), mainte- nance, and support. The design can involve many hundreds or even thousands of documents (specifications, schematics, drawings, etc.), each likely to be modified in some way during the project. Keeping track of all the changes and knowing the most current version of every item can be difficult. Thus, any project aimed at delivering a technical product should include provision to keep up with and control all this infor- mation; such is the purpose of configuration management. Configuration management represents policies and procedures for monitoring and tracking design information and changes, and ensuring that everyone involved with the project and, later on, the operation of the end-item has the most current information possible. Policies and procedures that form the configuration management system for a project should be included as a section in the quality plan. As with all procedures, the best configura- tion management system is whatever enables the desired level of control and is the simplest to implement. Two aspects of configuration management are configuration identification and configuration control. Configuration Identification Configuration identification is an inherent part of systems design that involves defining the structure of the system, its subsystems, and components. Mentioned in Chapter 2, any subsystem, component, or part that is to be tracked and controlled as an individual entity throughout the life cycle of the system is identified as a configuration item (CD). A Cl can be a piece of hardware, a manual, a parts list, a software package, or even a service. A subsystem that is procured is also treated as a CI. All physical and func- tional characteristics that define or characterize and are important for controlling the Clare identified and documented. Ultimately, every functional and physical element of the end-item system should be associated in some way with a CI, either as a CI on its own or as a component within a subsystem that has been identified as a Cl. Ideally, each Cis small enough to be designed, built, and tested individually by a small team. Master copies (electronic or paper) of the configuration documents for every CI are retained in a secure location (the “configuration center”) and managed by someone not involved in the functions of design, construction, manufacture, or maintenance. Chapter 9 Project Quality Management 341(Additional documentation such as the design premises, assumptions, and calcula- tions are not considered configuration documentation and are normally retained by the design authority.) Any modifications, waivers, or deviations to a CI are recorded so that all CI doc- umeniation reflects the “as-built” status of the system. In the case of an end-item such as a building, ship, or other one-of-a-kind system that becomes operational, the “as-built” specification will later be used for its operation and maintenance. Where multiple units are produced (such as cars, airplanes, appliances) and modifications and improvements are introduced over time, the specific configuration for each indi- vidually produced unit must be known, which requires that each specific CI in the product must be traceable to its specific “as-built” specifications. This is necessary, so that, for example, the correct spare parts, training, and operating manuals can be supplied, and problems can be traced and analyzed in the event of accidents, cus- tomer complaints, or claims regarding product liability. This concept of “traceabil- ity” was introduced in Chapter 4 and is illustrated in the following example Example 2: Traceability and the Apollo Spacecraft" 342 The way to establish the reliability of an item is to test many of them until one fails, or to design-in the reliability through methods of engineering analysis. Either way, everything about the item must be known—its manufacturing processes, the composition of it parts and materials, even the sources of those materials. For the Apollo space mission the goal of achieving mission success was set at 99, percent, and of crew survivability at 99.9 percent. To meet such high-level goals, every Cl (subsystem, component, part, etc.) as it moved through the design and manufacturing process was accompanied with a package of documents that established its genealogy and pedigree. The saying went, “If you ordered a piece of plywood, you wanted to know from which tree it came.” Half-inch bolts for the Apollo spacecraft involved an 11-step manufacturing process with certifica- tion tests at each step. Every bolt was subjected to rigorous testing, as were the steel rods from which they were made, the billets from which the rods were extruded, and the ingots from which the billets were forged. Everything about the process and tests for the bolts was documented, including the original source of the iron for the bolts—Minnesota—and even the mine and the mine shaft. This, extreme tracking and control is necessary to insure high reliability, and to enable problem diagnosis in case things go wrong. It comes with a price though, which is why bolts available for 59 cents at the hardware store cost $8 or $9 apiece on rockets and spacecraft Configuration Control Configuration control is the second aspect of configuration management; it is a tech- nique more for quality control than quality assurance but is covered here for the sake of continuity. The design of a system is normally specified by means of a large number of documents such as performance specifications, testing procedures, draw- ings, manuals, lists, and others that are generated during the design process. As the design evolves, these documents are subject to change, and an orderly scheme is needed to manage and keep track of all these changes. Such is the purpose of con- figuration control Configuration control is based on the following principles: 1. Any organization or individual may request a change—a modification, waiver, or deviation. 2. The proposed change and its motivation should be documented. Standard docu- ments exist for this purpose; for modifications, the document is called a change proposal or request, change order, or variation order. Part Ill Systems and Procedures for Planning and Control3. The impact of the proposed change on system performance, safety, and the envi- ronment is evaluated, as is its impact on all other physical items, manuals, soft- ware, manufacturing or construction, and maintenance. 4, The change is assessed for feasibility, which includes estimating the resources needed to implement the change and the change’s impact on schedules. 5. The change proposal is either rejected or accepted and implemented. The group responsible for making the decision, called a configuration board (CB) or a configuration control board (CCB), should include the chief designer as well as representatives from manufacturing or construction, maintenance, and other relevant stakeholders. Often the project manager or program manager chairs the group. 6. When a proposed change is approved, the work required to implement the change is planned. This includes actions regarding the disposition of items that might be affected by the change such as items in the inventory, equipment and processes used in manufacturing or construction, and manuals and other documentation 7. The implemented change is verified to ensure it complies with the proposed, approved change proposal. Change requests are sometimes classified as Class I or Class Il. Class I requests are for changes that can be approved by the contractor or the developer; Class TI changes must gain the approval of the client. Configuration control is an aspect of project control and, in particular, change control, both discussed in Chapter 11 Design Reviews Since the fate of an end-item is often sealed by its design, the project manager must insure that the proposed design is acceptable in all respects. This is the purpose of design reviews—to ensure that the users’ requirements and inherent assumptions have been correctly identified, and that the proposed design is able to meet those requirements in an appropriate way. Design reviews (not to be confused with gen- eral project reviews, described in Chapter 12) provide confirmation of the data used during the design process, design assumptions (e.g., load conditions), and design calculations. They should ensure that important life-cycle aspects of the product or end-item have been addressed and pose no unacceptable risks; examples of these aspects include: 1. Omissions or errors in the design Compliance to regulations, codes, specifications, and standards Cost of ownership Safety and product liability Reliability Availability Ability to be constructed or manufactured (manufacturability) Shelf life Operability 10. Maintainability 1. Patentability 12. Ergonomics SN ane The reviews should involve representatives from all disciplines, functions, and users who are or will be connected to the end-item throughout its life cycle and include outside designers and subject matter experts. (This relates to the concurrent Chapter 9 Project Quality Management 343344 engineering process, discussed in Chapters 4 and 13.) For example, a design review of a production facility (e.g., a chemical plant, mine, or factory) would include: + Representatives from the technical support area that will eventually be responsi- ble for maintenance of the facility. * People from the construction area or company that will build the facility. + Representatives from the marketing, procurement, legal services, and quality areas that in some way will occupy, make use of, or have to deal with the conse- quences of the facility. Early in the conceptual phase the re ht involve representatives from only a few functions, but as the project moves to later phases many more representa- tives will be involved. For the design of a simple part or component, a single review ‘upon completion of the design but preceding manufacture might be sufficient. In the case of a complex system, however, it will be necessary to convene several reviews at successive stages of the project. The quality plan should indicate when these reviews will be held, the stakeholders to be involved in each, who will chair them, and to what extent the designer is bound to comply with the reviewers’ directions or recommendations. The review dates, subjects, and attendees should be noted in the project communication plan. Formal Reviews Formal design reviews are planned events, preferably chaired by the project man- ager or someone else who is not directly involved in designing the product. For projects aimed at developing and delivering a product, the kinds of reviews com- monly include: 1. Preliminary design review: The functional design is reviewed to determine whether the concept and planned implementation fit the basic operational requirements 2. Critical design review: Details of the hardware and software design are reviewed to ensure they conform to the preliminary design specifications. 3. Functional readiness review: For high-volume or mass-produced products, tests are performed on early-produced items to evaluate the efficacy of the manufac- turing process. 4. Product readiness review: Manufactured products are compared to specifications to ensure that the controlling design documentation results in items that meet requirements Formal reviews serve several purposes: minimize risk, identify uncertainties, assure technical integrity, and assess alternative design and engineering approaches. Unlike peer reviews, the actual oversight and conduct of formal reviews are handled by a group of outsiders, although the project team accumulates information for the reviewers, These outsiders are technical experts or experienced managers who are inti- ‘mately familiar with the end-item and workings of the project and the project organi- zation, but are not formally associated with the project organization or its contractors. Since a formal review may last for several days and involve considerable prepara- tion and scrutiny of results, the tasks and time necessary to prepare and conduct the review and to obtain approvals should be incorporated in the project schedule. A prerequisite for the design review is thorough design documentation, so a common practice is to convene a “pre-review meeting” at which the design team gives the review team a brief overview of the design, documentation describing the design premises, philosophy, assumptions, and calculations, and specifications and Part Ill Systems and Procedures for Planning and Controldrawings for the proposed design. The review team is then allowed sufficient time (typically 14 days) to evaluate the design and prepare for the formal review meeting, Sometimes the review team uses a checklist to ensure that everything important is covered. In recent years the Internet has become an effective medium for conducting, design reviews and has reduced the cost of the reviews." Informal Design Reviews In addition to the formal reviews, the project manager should encourage many informal design reviews. These include informal discussions among designers, and between designers and other stakeholders. Good suggestions can come from different places, but it is up to the designer to decide whether or not to use them, Draft designs, reports, and other deliverables should be presented regularly and (ideally) voluntarily to peer designers and other stakeholders for informal review. Ina healthy quality culture, brainstorming is commonly used to evaluate and edit not only designs, but also reports and other deliverables of all kinds. The principle behind brainstorming is to freely generate as many ideas as possible, withholding any form of evaluation or criticism until after numerous ideas have generated. Only later are the ideas assessed and the good ones separated from poor ones. Example 3: Formal and Internal Reviews in the Mars Pathfinder Project"? Outside review boards conduct formal reviews for all of major NASA projects These reviews are important since termination or continuation of a project depends on the board's findings. Preparation for a formal NASA review can take an enormous amount of time. Senior project management for the Mars Pathfinder project (see also Example 8, Chapter 11) estimated that preparation for one such review, the critical design review, would require about 6 weeks of their devoted attention. This would divert time from the actual management of the project, which, paradoxically, could increase the likelihood of the project falling behind schedule and failing the review. To prepare for the review, project manager Brian Muirhead first ordered an internal review. Internal (or peer) reviews at NASA address @ narrow range of topics and require only a few days preparation. The main value of these reviews lies in making sure that everyone understands the decisions being made, that nothing is overlooked, and that the project is kept on track. Over 100 peer reviews were conducted during the 3 years of Pathfinder development. Tho Pathfinder internal reviow revealed a slew of problem areas, including lack of progress in defining system interfaces, rapid growth in the mass of the Mars lander (the prototype weighed too much), and a shortage of good engi- neers. These findings did little to inspire confidence about the project's ability to pass scrutiny of the design review board. The verdict from the critical design review is an all-or-nothing decision. The project earns either @ passing or failing grade. A failing grade initiates @ cancel- lation review, a process that can result in project termination. A project such as Pathfinder could be canceled if budget overruns ran as little as 15 percent. The Pathfinder design review board comprised 25 consultants and seasoned manag- ers from NASA and JPL (Jet Propulsion Laboratory, the site responsible for most of the Pathfinder design work), none of whom was associated with the project. Besides determining the future of a project, design reviews serve another purpose: to give it a kick in the pants. Preparation for the review sessions is a laborious endeavor and forces the project team to make decisions about unre- solved issues. Formal reviews may be held 3 or 4 times during the project. The review board for the Pathfinder critical design review was not happy with many aspects of the project; nonetheless, they did not initiate @ cancella- tion review. Instead, they approved the project, but instructed Pathfinder mana- gers to be more critical of designs, focus less on performance and more on cost, Chapter 9 Project Quality Management 345346 and stop obsessing over business innovations. These recommendations later proved useful and helped to make the Pathfinder project one of the most suc- cessful in the history of space exploration. The design review process is significant to quality, no matter how highly compe- tent the design staff. There is always more than one means to an end, and a designer can be expected to think of only some of them. Even the most competent people over- look things. Mature designers appreciate the design review process in terms of the networking experience, innovative ideas provided by others, knowledge gained, and reduction of risks, but less mature designers tend to feel insulted or intimidated by it. It is human nature for people to be less than enthusiastic about others’ ideas and to resist suggested changes to their own. The design review process seeks to achieve “appropriate quality” (a balanced compromise agreed upon by the stakeholders) and refrains from perfecting minor features and faultfinding. Review meetings are also discussed in Chapter 12. Audits Unlike design reviews, which relate only to the design of a product, audits have broader scope and include a variety of investigations and inquiries. The purpose of audits is to verify that management processes comply with prescribed processes, pro- cedures, and specifications regarding, for example, system engineering procedures, configuration management systems, contractor warehousing and inventory control systems, and facility safety procedures. They are also performed to verify that techni- cal processes such as welding, adhere to prescribed procedures, and to determine the status of a project whenever a thorough examination of certain critical aspects of the work is required. Any senior stakeholder such as a customer, program manager, or executive can call for an audit. Like formal design reviews, audits are relatively for- mal and normally involve multifunctional teams. Unlike design reviews where some- times innovative ideas originate, audits focus strictly on verifying that the work is being performed as required. They are performed by internal staff or by independent, external parties who are deemed credible and, ideally, unbiased, fair, and honest Preparation for an audit includes agreement between the auditor and stak: holder requesting the audit as to the audit's scope and schedule, and the responsibi ities of the audit team. The audit team prepares for the audit by compiling checklist and sometimes attending training sessions to learn about the project. Each auditor is required to prepare a report within a few days following the investigation detailing any nonconformities found, rating the importance of the nonconformities, describ- ing the circumstances under which they were found and the causes (if known or determinable), and providing suggestions for corrective action. While the focus is on uncovering nonconformities, commendable activities are sometimes also noted in the audit report. A typical thorough audit will take 1 to 2 weeks Classification of Characteristics Asystem (deliverable or end-item) is “specified” or described in terms of a number of attributes or characteristics, including functional, geometrical, chemical, or phys cal properties and processes. Characteristics can be specified or described in terms of numerical specifications, which often include tolerances of acceptability. In a com- plex system there are typically a large number of characteristics defined on draw- ings and other documents. As a result of the Pareto principle (which states that, in general, the large majority of problems in any situation are caused by a relatively small number of sources), the most cost-effective approach to quality assurance is to Part Ill Systems and Procedures for Planning and Controlattend to the system or component characteristics that have the most serious impact on quality problems or failures. This does not mean to imply that other characteris- tics should be ignored, but rather that limited resources and activities for inspection and acceptance testing should be directed first at those items classified as most cru- cial or problematic. Characteristics are typically classified into four categories: critical, major, minor, and incidental (or, alternatively, critical, major A, major B, and minor). The critical classifica- tion is reserved for characteristics where a nonconformance would pose safety risks or lead to system failure. Quality plans often specify that items with critical characteris- tics be subjected to 100 percent inspection. The major classification is for characteristics where nonconformance would cause the loss of a major function of the deliverable. The minor cla: ation is for characteristics where nonconformance would lead to small impairment of function or to problems with manufacturability or serviceability. Characteristics classified as incidental would have minimal effect or relate to relatively unimportant requirements. The classification assigned to a characteristic is determined by the designer of the system in collaboration with others such as the designer of the next higher-level system, designers of interfacing systems, or staff from manufactur- ing or construction. Together they analyze the design characteristics regarding safety and other requirements, and classify them using a set of ground rules. Classification also applies to kinds of nonconformities or defects, but this should not be confused with the classification of characteristics. In welded structures, for example, the specified characteristics often include the “absence of any cracks or of certain amounts and kinds of impurities” in the weld metal. A crack (a nonconform- ity that could lead to a catastrophic failure) would be classified as “very serious,” whereas a small amount of an impurity in the weld (a nonconformity that would have no effect on the integrity of the structure) would be classified as “minor.” Classification of characteristics serves as a basis for decisions regarding mod fications, waivers, and deviations at all levels of a system. For example, classifica- tion of characteristics in a higher-level system provides guidance to designers of the lower-level subsystems and components that comprise the system. Classifying the braking performance of an automobile as critical (e.g., that the automobile when traveling at 25 miles per hour should be able to stop within 40 feet on dry pave- ment) tells the braking system designers that components of the brakes should be classified critical as well. Failure mode and effect analysis (FMEA), discussed below, sometimes plays an important role in this classification process. Sometimes the characteristic classifications are listed in a separate document, although it is more practical to indicate the classifications directly on drawings and other specifications by means of symbols such as “C” for critical, “Ma” for major, “Mi for minor, and so on. Absence of a symbol normally indicates the lowest prior- ity, although some organizations denote even the lowest classification with a sym- bol as well. Only a small percentage of characteristics should be classified as critical. Too large a number of characteristics classified as critical could be the sign of poor design: if everything is critical, nothing in particular is critical! Failure Mode and Effect Analysis Any system can potentially fail as the result of a variety of conditions such as the short-circuiting, cracking, collapsing, or melting of its components, or inadequate, missing, or incorrect steps and procedures in its design production, or operation. FMEA (also called failure mode, effect, and criticality analysis, FMECA) is a technique to determine in what ways a technical system might fail, and what effects the identified failures would have on the system’s performance and safety, and on the environment. Chapter 9 Project Quality Management 347348 FMEA is a procedure normally used during the early stages of system develop- ment; it involves the following steps: 1. List the relevant components of the system. 2. Identify all the possible ways in which the component or system might fail (the {failure modes). This is best done by a team that brainstorms all the conceivable failure modes. The causes of each failure mode and conditions under which they are likely to occur are also listed. 3. Assess the probability of each failure mode occurring. 4, Describe and assess the probable effects (or impacts) of each failure mode; these are impacts on the performance and safety of the system, and on the environment. 5. Assess the severity or seriousness of the effects. 6. Rate the criticality of each identified failure mode. Criticality is a function of both the probability of the failure and the seriousness of the effects. 7. Prepare a plan to circumvent the failure mode and /or mitigate the effects of the failure. Where applicable, describe an appropriate response in case the failure occurs. In cases where conformance to a specific characteristic is necessary to prevent a particular failure, the characteristic is classified as critical. Table 9-1 illustrates: The columns “Sev” (severity), “Prob” (probability), and “Det” (detectability—potential failure will be easy or difficult to detect) are each rated 1 to 10 for each of the potential failure modes. RPN (risk priority number), which is the criticality of the failure mode, is: RPN = Sev X Prob x Det Items are categorized by RPN and the highest-priority ones are addressed first. Although a failure by itself might not be critical, combined with other failures it could have a very serious effect. The Chernobyl disaster is an example where a chain of errors (each alone not very serious) led to a catastrophic failure—the meltdown of a nuclear reactor. Thus, FMEA must consider combinations of possible failures modes as well individual failure modes. Although used primarily in design and engineering analysis, FMEA can also be used to identify issues affecting project costs and schedules—a tool in project risk management, described in Chapter 10. Modeling and Prototyping Designers use a variety of models to ensure the quality of their products, including computer simulation models, mathematical models, three-dimensional scale mod- els, and full-scale prototypes, each to gain a better impression of how the final prod- uct, system, or subsystem will look and perform. Models and prototypes also serve a marketing role by helping others to understand and see a “vision” of the product or system. A full-scale wooden or plastic mock-up of the driver's cab of a new truck or the cockpit of a new airplane, for example, helps the producer to sell its products and to obtain suggestions or criticisms about features and configurations. In product development projects, constructing models helps minimize the risk of failure to meet technical requirements. As shown in Table 9-2, it is customary in development projects to build and test different types of models coinciding with project phases to eliminate different risks. (Table 9-2 applies to development projects where the deliverable is the detailed definition and specification for the product and its production process, but not the manufactured product itself; hence the absence of a “production” or a “building” phase.) Part Ill Systems and Procedures for Planning and Controlore Table 9-1 FMEATabie Subsyeem Faure Mode and Elect Anais Nee Component (esi FMEA) Tet Dae Design Lead Key Oat Revision Date Page ot Syste Core Team Port espana & } tz] GR | Ir “fe |Pece | canton i stenvFurcton acion oxen]Table 9-2 Phases of equipment development. Opyrcrives Ratarine 70 ‘Tite ELIMINATION, Project Putase Monet. Burtt aNp Test oF Risks Risks EniMiNaTeD Concept Exploratory development Proof that the concept The risk that the concept ‘model (XDM) would be feasible ‘would not be feasible (Breadboard models) Such models could be built for the entire system or for specific high-risk subsystems Validation Advanced development Proof that the ‘The risk that the performance model (ADM) product would of the system and its perform according _ interfaces with other systems to specifications and would not be acceptable interface well with other systems (form, fit and function) Development Engineering Proof of reliability, ‘The risk of poor operational development model availability, and availability or reliability (EDM) manufactured maintainability from the intended final materials Ramp-up Pre-production models Proof that the ‘The risk of unforeseen (PPM) product could be problems in manufacturing ‘manufactured reliably in the production facility and could be deployed effectively Projects for the development of new chemical or metallurgical processes often include similar project phases and models (Table 9-3). ‘The models for such processes usually start out as laboratory equipment but ultimately grow in sophistication and capacity to enable a pilot operation, then ultimately a demonstration plant that closely replicates the proposed facility. The kind of models used in a project, whether full-scale physical mock-ups, scale models, mathematical models, computer simulation models, breadboards, or proto- types, depends on the information needed versus the expense of creating and using them. For a small product comprising only a few components, building and testing Table 9-3 Phases for development of chemical or metallurgical development. Project Puase Ossecrive Laboratory experiments To prove the basic concept. Pilot plant To learn how the process works when scaled up. This provides inputs for the design of the final plant. Demonstration plant To provide a full-scale plant that demonstrates to potential customers the economic feasibility as well as operational aspects. 350 Part Ill__ Systems and Procedures for Planning and Controla full-scale model that closely resembles the final product is probably cost-effective; for a complex system with innumerable components, it usually is not and computer simulation and mathematical models are normally more effective. Example 4: Modeling the Form and Fit of Boeing 777 Components"* (One of the most pervasive problems in the development of large aircraft is align- ing vast numbers of parts and components so that during assembly there is no interference or gaps between them. In the mid-1980s Boeing invested in three- dimensional CAD/CAM (computer-aided design/computer-aided manufacture) technology that would enable designers to see components as solid images and simulate their assembly into subsystems and systems on a computer screen, By 1989 Boeing had concluded that “digital preassembling” of an airplane could sig- nificantly reduce the time and cost of rework that usually accompanies introducing a new airplane into the marketplace. In 1990 it launched the Boeing 777 twinjet program and began involving stakeholders such as customers, design engineers, tool makers, manufacturing representatives, and suppliers in the concurrent engi- neering design process (see Example 7, Chapter 4). The physical geometry of the airplane's components was determined with CAD/CAM technology instead of with physical mock-ups that are time-consuming and expensive to build. As @ result, the 777 program exceeded its goal of reducing changes and rework by 50 percent. Testing of Prototypes and Models As shown in Table 9-2, a goal of building and testing models is to systematically reduce the risk that the final product will not be satisfactory. Models enable data to be gathered about systems and subsystems that will assist in later stages of devel- opment. Experiments with models reveal design shortcomings. Data about stresses and strains in components, for example, provide information about potential fail- ure modes in the final product. (Occasionally, the engineers and technologists who design and build models become attached to them and do not want to see them get broken or damaged, and they actually resist doing tests on them. Of course, the objective of modeling is not to create perfect models but to generate data that will result in a high-quality final product.) While early tests on components to acquire technical data should be supervised by designers and test personnel, functional tests on full-scale models should involve people who most resemble typical users. For products that are to be manufactured in quantities, the design should be veri- fied by means of a qualification test before manufacturing ramp-up. The qualification testing process happens in the opposite way of the system design process. While typ- ically the design process happens top-down, starting with the overall system design and cascading down to the design of individual subsystems and components, qualifi- cation testing happens bottom-up, starting with the testing and qualification of indi- vidual components, then of subsystems, and finally of the full, completed system. 9.4 PRoceEsses AND TECHNIQUES FOR QUALITY CONTROL When a development project terminates with handover to an operational process (cg, repetitive manufacture of a product), the deliverables of the project include the product design, the production process design, and a plan or specification to insure the production process will provide repeated delivery of a quality product. The last of these is the subject of quality control, which concerns anything that must be done Chapter 9 Project Quality Management 351352 to insure that the output of a repetitive process remains at the required, specified quality level and does not deteriorate Inspection and Acceptance Testing of the Final Product Quality control of items produced repetitively includes inspections on the product and its components throughout the production process. Characteristics classified as critical are always inspected, whereas minor or incidental characteristics are not. In automobile production, for example, the braking and steering performance on every vehicle is tested. When items are produced in large quantities, some of them might be subjected to destructive tests; when only one or a small batch of an item is pro- duced, the inspection and testing procedures must involve non-destructive methods such as radiographic, infrared, and ultrasonic imaging of critical components. For certain critical components produced in high volume, sampling is com- monly used to reduce the cost of inspection. Based on the results of tests from a few samples, a statistical inference is made about the quality of the entire batch or proc- ess. Obviously, this method is mandatory when the testing destroys the product. To check the heat treatment of engine blocks, for example, a piece must be cut from a heat-treated engine block and tested in a laboratory. Testing of samples is common whenever a process produces a large number of identical products. Although testing of the end product from a production process does not lie within the realm of project quality management, per se, the testing procedures and other quality assurance processes to be used during production should be speci- fied during any development project where the result is a mass-produced item, and included as project deliverables. Product designers—who have intimate knowledge of the key characteristics of the product and its components—are well suited to spec- ify the ways those components should be quality checked once production begins. Basic Tools of Quality Control In 1982 professor Kaoru Ishikawa of the Tokyo University defined what have come to be known as the “seven basic tools’ or “magnificent seven” of quality control:”!5 1, Check sheet 2, Flowchart 3. Run chart and control chart Scatter diagram Pareto diagram Histogram 7. Cause-and-effect diagram Although Ishikawa worked in a production environment (Kawasaki Steel Works) and not a project environment, most of the seven basic tools are applicable in every- day situations and some in particular to projects.'® Ishikawa’s tools are aimed at identifying the sources of defects and nonconformities in products and processes, but are equally applicable for identifying sources of and resolving all kinds of poten- tial problems, including those associated with project risks. The application of some of the tools to project risk analysis and management is discussed in the next chapter. Check Sheet A check sheet is a sheet created especially for collecting data about a problem from observations. The content and format of the sheet are uniquely designed by the team investigating the problem. Data recorded on the sheet is subjected to analysis using the other six tools. A check sheet should not be confused with a checklist, the latter Part Ill Systems and Procedures for Planning and Controlbeing a list of steps, issues, or pointers based upon prior experience to be considered (eg,, in planning a project). Flow Chart Flowcharts show the steps in a procedure and their relationships. Process flowcharts show the steps or tasks in a process. Project networks are a form of flowcharts that show the sequence of activities in a project. For problem analysis, usually more detailed flowcharts are needed to reveal the steps and tasks within the activities. An example is the diagram showing material flow in Figure 3-5. Close scrutiny of a flowchart often can reveal the steps or relationships that cause quality problems. Run Chart A run chart is a graph of observed results plotted versus time to reveal potential trends or anomalies. The plot of schedule performance index versus cost perform- ance index as illustrated in Figure 11-16 is a form of run chart that tracks project performance and indicates whether a project is improving or worsening in terms of schedule and cost. Control Chart and Scatter Diagram Control charts and scatter diagrams are widely used for tracking and control of repetitive events (e.g., mass production). For projects that include the development of production processes, however, one of the deliverables would be specification of the relevant charts for controlling the quality of the process. Readers involved in projects aimed at the delivery of continuous operations systems should refer to books on statistical control techniques, such as Juran’s Quality Control Handbook.”” Pareto Diagram and Histogram Vilfredo Pareto, a nineteenth century Italian economist born in Paris, formulated “Pareto's Law” of income distribution. This law states that the distribution of income and wealth in a country follows a regular logarithmic pattern: 20 percent of the peo- ple own 80 percent of the wealth. This principle, also dubbed the “80/20 rule,” has since been found to apply in principle to a wide variety of situations, including those relating to quality. Quality consultant Dr. Joseph Juran in the late 1940s posited that the large majority of defects are due to a small minority of the causes and, thus, for economic reasons it makes sense to separate the vital few causes of defects from the trivial many, and to direct improvement efforts to those few. Figure 9-3 is a Pareto diagram. The histogram in the bottom part of the diagram shows the number of defects versus kinds (sources) of quality problems; the line in the top of the figure represents the cumulative effect of the problems corresponding to scale on the right. As shown, the first sources account for roughly 43 percent of the problems; the first and second combined account for roughly 70 percent. Thus, resolving just the first two problems would eliminate 70 percent of the problems. In project environments, Pareto analysis would be used to track the sources of recurrent problems, and to identify those causing the most problems and most in need of attention. Cause-and-Effect Diagram Quality problems and risks are often best addressed through the collective experience of project team members. Team members meet in brainstorming sessions to generate ideas about problems or risks. These ideas can be recorded on a cause-and-effect (CE) diagram (also called a fishbone or Ishikawa diagram), which is a scheme for arrang- ing the causes for a specified effect in a logical way. Figure 9-4 shows a CE diagram Chapter 9 Project Quality Management 353354 8 70 ¢ 8 Bo e E 33 40 ce i a3 8 oo 32 3 i 3 ge 2 20 2 a 10 ° Figure 9-3 Kinds of problems Pr Pareto diagram. Figure 9-4 CE (fishbone or Ishikawa) diagram. to determine why a control system does not function correctly. As the team gener- ates ideas about causes, each cause is assigned to a specific branch (e.g., “assembly procedures” on the Quality of Assembly branch). CE diagrams and brainstorming can be used in two ways: (1) given a specified or potential outcome (effect), to iden- tify the potential causes and (2) given a cause (or a risk), to identify the outcomes that might ensue (effects). CE diagrams do not solve problems but are nonetheless useful for identifying problem sources and planning actions to resolve them. The seven basic tools are relatively simple. Following are two techniques that are more sophisticated, Part Ill Systems and Procedures for Planning and ControlOtner projects +, Workload of ———™ designer Maifunctoning ot (7 control system prototype Attention to assembly Quality of procedures ‘components Quality of + assembling Vendor AQ Figure 9-5 Casual loop diagram for control system problem, Causal Loop Diagram As the name suggests, a causal loop diagram shows the causes of a certain problem or situation. It can be used to illustrate the structure of a complex system and the influ- ence that variables in the system have on one another.* ‘As shown in Figure 9-5, variables in a causal loop diagram are connected by arrows to indicate cause-effect relationships. The positive and negative signs indicate the direction in which the variable at the arrow’s head changes when the variable at the arrow’s tail changes. That is, a positive sign indicates a reinforcing effect—the variable at the arrow’s head increases as the variable at the arrow’s tail increases. A negative sign indicates that the variable at the arrow’s head decreases when the variable at the arrow’s tail increases. For example, in Figure 9-5 when the number of projects increases, the designer’s workload also increases, and when that happens the designer's attention to assembly procedures decreases. Causal loop analysis is a way of modeling the dynamics of a complex system, but it involves considerations beyond the scope of this book. In many cases the more superficial analysis afforded by CE diagrams is sufficient. Current Reality Tree A current reality tree (CRT) is a technique for analyzing an existing situation or sys- tem. It starts with an identified (observed) undesirable effect (UDE) or symptom, and then is used to identify relationships between the UDE and other undesirable effects. Unlike CE and causal loop diagrams, a CRT also considers whether the causes iden- tified are sufficient to have resulted in the specified UDE. This “hard logic” requires that assumptions about the situation be identified, and that as many facets of the problem as possible be uncovered, which leads to the identification of underlying, catises (root catuses or core problems)—much in the same way that symptoms in a patient lead to diagnosis of health problems.” For example, suppose a control system is malfunctioning, and one possible source is the procedure used to assemble the system (UDE 900). Figure 9-6 shows the CRT for the situation; it is read from the bottom to the top as follows: Entities 100, 200, 300, and 400 are causes for the UDE numbered 500. The oval around the Chapter 9 Project Quality Management 355Example of a CRT. arrows indicates that these four entities are sufficient to have caused UDE 500. In the same way, entities 500, 600, 700, and 800 are sufficient to have caused UDE 900. The CRT approach requires more effort than simple CE analysis and, hence, would be applied only to problems that are considered more severe (the first 20 percent of the causes that lead to 80 percent of the problems). The CRT is one of several tech- niques and tools described in the literature on theory of constraints, which include the future reality tree (describes the desired future situation after a problem has been resolved), the prerequisite tree (a process for overcoming barriers), and the transition tree (a process for problem solving).”” Other Tools for Quality Assurance and Control Mentioned elsewhere in this book are planning and control methods that also apply to quality assurance and control. For example, much of the quality assurance effort in a product design project is directed at keeping the project team focused on cus- tomer requirements, and making sure that those requirements do not become dis- torted or misinterpreted as the project moves from stage to stage and the work changes hands. Quality function deployment (QFD) discussed in Chapter 4 is a method for defining customer requirements and ensuring that those requirements remain in the focus throughout the design and production process. Likewise, checklists as described in this book for preparing plans, assessing risks, and monitoring work progress help maintain quality by assuring that impor- tant issues or items are not overlooked. These checklists can be used for inspections, testing, design reviews, and FMEA. A potential disadvantage of checklists is that people rely on them too much and ignore issues or items not on the list. The last item on every checklist should be “Now, list and scrutinize all perceived important items not on this checklist!” 356 PartIll__ Systems and Procedures for Planning and Control