Reliability Specification & Test RST

Reliability Specification & Test Manual

Page 1 of 20

Reliability Specification & Test RST

Reliability Specification & Test - RST

1.0 Introduction
The cost and difficulty of implementing change during a design and development programme both increase as the programme progresses (usually by ten fold from phase to phase from design, to development, to production and to service). It is therefore more effective to make changes as early as possible to gain the most from the finite resources people, time, or money. The recommended engineering change profile is shown in Figure 1. But to achieve this profile requires a disciplined approach to the New Product Introduction (NPI) process that includes the methodology for specifying reliability targets, and carrying out appropriate and effective test programmes.

Figure 1. Engineering Change Profile A common occurrence is for quality problems to surface late in the project (or worse by being identified in the hands of the customer) in spite of many thousands of hours and miles of testing or the so-called miles for smiles approach. The test programme needs to be customer focussed with all test activities being aimed at preventing customer problems being identified too late in the programme. By identifying unforeseen design defects early in the development cycle and providing
Page 2 of 20

Reliability Specification & Test RST

rapid and efficient corrective action, reliability growth may be achieved. To achieve this requires a distinction to be made between development and validation testing.

2.0

Advanced Product Quality Planning APQP

The Advanced Product Quality Planning (APQP) approach and its linkage to the NPD process is shown in Figure 2, which clearly defines the framework for APQP in terms of the five major stages that must be instigated and completed by each of the NPD gateways. It is vital to incorporate feedback loops to ensure quality growth as the project progresses through the NPI process. NPI Process Gateways

DR0

DR1

DR2
Concept and Programme Approval

DR3

DR4

DR5

VOL

Strategy Review

Concept Selection

Design Release

Production Release

Project Review

Plan and define programme

Product design and development

Process design and development

Product & Process Validation Feedback, assessment and corrective action

Figure 2. Advanced Product Quality Planning (APQP) and the NPI Process Table 1 shows the various APQP activities for each of the five stages, for which the detailed inputs are shown as a timing plan Appendix 1. APQP STAGES Plan and define programme Product Design & Development APQP ACTIVITIES Customer input requirements Quality and Reliability Goals Product Assurance Plan Design FMEA

Page 3 of 20

Reliability Specification & Test RST

Process Design & Development

Product & Process Validation

Feedback, assessment & corrective action

Design Reviews Design Verification Plan Prototype Build Control Plan Facilities, Tools and Gauges Process Flow Charting Process FMEA Measurement System Evaluation Process Capability Study Plans Pre-launch Control Plans Production Trial Run Process Capability Studies Production Control Plan Product Validation Testing Quality Planning Sign-off Analyse and reduce variation Monitor product performance Feedback data to project team Apply disciplined problem solving

Table 1. Advanced Product Quality Planning (APQP) Stages and Activities

3.0

Reliability Specification and Test (RST) Process

This Reliability Specification and Test RST process is aimed at recommending best practice in setting reliability predictions, planning test programmes and analysing the results. The process introduces the activities during product design and development in a flow chart format that will lead to the desired levels of component reliability, at the end of the engineering phase of a product programme (DR3 gateway in the NPD process). Therefore, this process concentrates on the front-end activities of the programme rather than on the later validation and quality maturation phases. As such, the RST process directly supports the requirements for Advanced Product Quality Planning (APQP). Figure 3 shows a Reliability Management and Control Document (RMCD) that incorporates the tools and techniques that must be used to ensure that the product is designed, developed and produced with quality, reliability and maintainability as key determinants of the success of the product in the market place. A fundamental aspect of the RMCD is to create a vehicle level plan that demonstrates linkages between the functional specification and quality growth, through the identification of reliability critical items and processes and application of system FMEAs to drive the development testing programme through to validation and quality planning sign-off.

Page 4 of 20

Reliability Specification & Test RST

Drawn by:

CT/SMM

RELIABILITY MANAGEMENT & CONTROL DOCUMENT

J F M A YEAR 1 M J J A S O N D J F M A YEAR 2 M J J A S O N D J F M A YEAR 3 M J J A

Issue: 1 Date: 27-03-02

YEAR ACTIVITY
Phases

O
Ramp-up

Product Strategy & Planning up to DR0 then Concept Evaluation

Concept Development & Business Case

Product Design & Validation

Productionisation

Pre-production

DR0
Gateways
Strategy Review

DR1
Concept Selection

DR2
Concept & Project Approval

DR3
Design Release

DR4
Production Release

DR5
Volume Release

VOL
Project Review

Benchmarking & target setting Functional Specification Reliability Critical Items/Processes (RCIs & RCPs ) Reliability predictions based on Minimum Life process Design & process FMEAs Design for Manufacture (DFM/A) Test plans Development testing Validation testing Process capability studies Reliability Growth Quality Planning Sign-Off

Competitor Vehicles

Formalise

Reliability Specification & Test - RST

FMEAs at System Level Apply on New Designs

Test to Failure Design Proving In-House and at Supplers Derived from Test Results

Figure 3. Reliability Management and Control Document (RMCD) The objectives of the Reliability Specification and Test (RST) can be summarised as follows. Use Customer Focussed Reliability Specifications This means being aware of what matters to the customer and then ensuring that the specification reflects this. This does not mean that all reliability specifications need to be increased from current levels contained in the Tata Engineering Standards, as it may well be possible to reduce certain specifications without affecting customer satisfaction. Early Design Analysis Drives the Development Programme Testing should be focussed on exploring areas of potential weakness in a design. It makes sense to analyse the design before testing to highlight potential areas of weakness, or areas of uncertainty, and then to plan the test work. Focus on Product Improvement throughout the NPI Process The objective of the Reliability Specification and Test RST process is to improve the product at every phase of the design and development programme by focussing design and development added value work. That is, work that changes the product so it will perform better in the hands of the customer. Validation work, which involves measuring where we are, does not change the product and so has no impact on the customer. Figure 4 shows the concept of linking the initial design to meet the functional specifications through system FMEAs, which drive the development testing programme to quality planning sign-off. Figure 5 shows how this is achieved in practice through application of the Reliability Specification and Test (RST) process.

Design to meet functional specification Page 5 of 20

Reliability Specification & Test RST

Determine Reliability Critical Items and Processes (RCIs/RCPs)

List of RCI/RCPs

Project team sets vehicle minimum and maximum life in years

Calculate Minimum Life for RCIs and all major vehicle systems

Minimum Life at system level

Convert Minimum Life to number of cycles and Km

Road load and customer mileage data

Start system design FMEAs and identify design changes at concept stage of NPI process

Start process design FMEAs

Use system level FMEA RPNs to prioritise sub-system/component FMEAs

Create vehicle test plans and cascade to system and component level

Test Plans

Quality planning sign-off

Figure 4. Process for Linking System FMEAs and Development Testing

Decide component reliability specification

YES
Component Design

NO

Does the specificatio n need change? 6 of 20 Page

Reliability Specification & Test RST

Analyse design & process to compare analysis results with specification Identify weaknesses and risks

Improve design and/or process?

YES
Implement improvements

NO NO
Is testing required? No dedicated test

YES
Use analysis results to drive and focus development test programme Test to failure by exploring the limits of performance and life

YES
Have test objectives been met? Design Sign-off

NO
Analyse cause of fault/failure

Does design require improving?

YES
Implement improvements

NO

4.0

RST Major Process Steps

Continue testing to generate new failures

Figure 5. RST Process

This guideline document has so far introduced the basic concept behind the RST process. The remainder of the document describes each of the major steps shown in Figure 5. A PowerPoint presentation is available and presents the sub-processes behind each of the major steps, and as such it is intended to be used in conjunction with this guideline as shown in Appendix 2.

Page 7 of 20

Reliability Specification & Test RST

4.1

Decide Component Reliability Specification

Traditionally the main measure used for predicting reliability has been Faults Per Vehicle (FPV) and Time to First Failure (TTFF). The RST process uses the Minimum Life concept for setting the reliability target. This approach relies on the design strength distribution (due to manufacturing variation) being optimised so that zero overlap is achieved with the stresses imposed by customers during product use (different patterns of customer usage). Over time the strength of components tends to reduce due to exposure to stresses, which can lead to an overlap condition with customer imposed stresses, and hence product failure the time taken for this situation to arise is the minimum life of the component. The algorithm for calculating the minimum life is shown in Figure 6.

E = A [ B + C + (0.5D) ]
A = Severity Rating Scale (1 to 10) B = Cost of Repair Rating Scale (1 3 5) C = Time Off Road Rating Scale (1 3 5) D = Activity Rating Scale (1 3 5) E = Minimum Life (1 to15 is short 16 to 35 is medium over 35 is long) The method for establishing the ratings for A to D is shown in Appendix 3. Figure 6. Algorithm for Calculating the Minimum Life The minimum life is then converted to an appropriate measure for the component or system miles, number of operations, time etc. This is achieved by using typical customer usage drive cycles for the different product segments and markets. For example a car based vehicle with a three year warranty and a ten year design life would typically equate to: short is four years medium is seven years long is ten years. The assumptions for a commercial vehicle would be different. Appendix 3 shows the method for establishing the Minimum Life with a worked example. 4.2 Component Design

This is the opportunity to develop a robust design, which meets a number of important design criteria: cost, weight, reliability specification etc. The technical targets for each of the components and systems are derived from the QFD process output product profile. 4.3 Analyse Design and Process

After creating an initial design, it should be analysed and improved as much as possible before any test planning is carried out. The principal approach is to use the Failure Mode & Effects Analysis (FMEA) technique at system level rather than at individual component levels. The system level FMEAs provide a prioritised view of the reliability critical components that require an individual FMEA. Other analysis tools that are typically
Page 8 of 20

Reliability Specification & Test RST

used include Fault Tree Analysis (FTA) and Finite Element Analysis (FEA). Significant sources of data during this stage include past warranty incidences and typical customer usage profiles from the QFD process. Each of the failure modes from the FMEA should be ranked using the Risk Priority Numbers (RPNs) to determine the priority for, and focus development testing. The main output from the FMEA approach is to identify design improvements prior to prototype build and subsequent testing, in order to realise rapid and early engineering changes as shown previously in Figure 1. This element is managed at the system and aggregate Design Reviews. 4.4 Is Testing Needed?

Here, it is important to understand that the task is to carry out development testing not validation testing, which is carried out later with product built off production tools and processes. The decision to test or not should be based upon a number of important criteria, which support two major factors: meeting customer requirements, and managing risk. The customer requirements in terms of the component reliability specification are defined by the minimum life, and risk is assessed as a result of design analysis and existing knowledge of the design. When a level of confidence already exists it may be decided that existing test results are acceptable, and no additional testing is required. The early involvement of test engineers in this process is essential in determining the focus of the test programme. When the nature and level of testing has been determined and approved by the project team, the test programme can be created. The results of the design analysis (FEA and FMEA) must drive and focus the test programme to ensure an efficient and value added development programme. The main steps are as follows: Determine failure modes and critical causes from the FMEA output Determine the linkages of failure modes and causes primary and secondary Determine the optimum test order environmental, legislative, load Determine the level of testing component, system, vehicle Set test stress levels based on customer usage With a good understanding of the failure modes and the causes of the failures the life specification can be considered, which may vary considerably. Shorter life specifications can be tested relatively quickly whilst long life items present difficulties due to the time taken to complete a test. There are two methods of dealing with this problem: increase the frequency of inputs into a shorter time frame, the other is to raise the level of inputs such that the component or system is over stressed for a period of time that represents the life requirement and beyond until failure occurs. If a failure does not occur within a realistic time after the specified life requirement the test can be terminated. The time at which a test can safely be terminated will vary depending on the component or system under test and the type of test it is undergoing.

Page 9 of 20

Reliability Specification & Test RST

Margins on safety critical components or systems should be greater than on noncritical items, and the engineer must consider a maximum life target for the component and declare it as part of the test plan. With both of these approaches it is important to recognise that accelerating a test does not necessarily accelerate every failure mode. Therefore it is essential to determine what needs to be achieved from any particular test and plan for it accordingly. Appendix 3 shows a document for recording the steps for an FMEA test process. In order to assist project teams with the nature and level of testing there are three tools. The Test Category Matrix shown in Table 2 provides guidance relating to minimum life expectations and test costs. The Test Requirements Matrix shown in Table 3 recommends the type of testing and test sample size. The Test Sample Size shown in Table 4 provides guidance for selecting the number of components for the large, medium, and few categories from Table 3. In the handbrake example from Appendix 2 the test category from Table 2 is 1 (low cost medium minimum life), the test requirements from Table 3 are to test a medium number of samples in all environments taking in to account crucial parameters (test category 1 medium risk). Therefore, the test sample size is 9 as derived from Table 4.

TEST COST LONG LIFE MEDIUM LIFE SHORT LIFE Low component 1 1 2 Medium system 2 2 2 High vehicle/body 2 3 3 Table 2. Test Category Matrix for Cost and Life

Design Risk

TEST CATEGORY

HIGH RISK New Design, New Process, Poor or variable known Process, Unknown environmental effects, High RPN Test large no. of samples in all environments taking in to account critical parameters Test large no. of samples in all environments taking

MEDIUM RISK Modified Design, modified process, encouraging in-service data, unsure of environmental effects, medium RPN Test medium no. of samples in all environments taking in to account crucial parameters Test medium no. of samples in crucial environments taking in

LOW RISK Blueprint design, good/stable known process, known environmental effects, low RPN Test medium no. of samples in crucial environments taking in to account crucial parameters Test low no. of samples in crucial environments

Page 10 of 20

Reliability Specification & Test RST

in to account crucial to account crucial taking in to account parameters parameters crucial parameters Test medium no. of Test few no. of No dedicated tests samples in crucial samples in crucial monitor other test environments taking environments taking in results in to account crucial to account crucial parameters parameters Table 3. Test Requirements Matrix for Category and Risk LARGE MEDIUM FEW

15
4.5 Test to Promote Failures

9
Table 4. Test Sample Size

During the development phases of the NPI process, testing should aim to promote failures, rather than a passive pass-the-test engineering culture. Corrective action should be taken to cure the root causes of the failure modes to improve the product. Testing to a specification (validation testing) should only be done after engineering design and development is complete, at the DR4 gateway. The worst realistic case stress level for testing (baseline stress level) should be established for typical customer usage by considering what is meant by high stress for a particular component or system. It may well be that a stop-start driving cycle will cause more failures rather than heavy duty or maximum power. Combinations and levels of stress will influence failure modes, and it is the interaction between the combination of stresses and levels of stress (as well as the operating environment) which is important. Accelerated testing aims to reduce the duration of testing by promoting test failures more quickly through the use of higher stresses. The test requirements matrix from Table 3 will have identified the number of test samples, which should be individually subjected to different stresses between the baseline and maximum stress levels. The testing should be implemented with a positive aim to promote failures, to analyse their cause, and to use the information to introduce improvements. Figure 7 shows a plot of three different failure modes, which have been recorded against individual test samples for accelerated stress levels up to +100%. This demonstrates levels of: either safety margin at the minimum life level, failure at the minimum life, and failure within the minimum life. The following should be considered: does the safety margin mean levels of over-engineering or are design changes required to eliminate failures within the minimum life 100% stress level. As a general rule, the greater the FMEA severity rating of the failure mode, the larger the reliability stress margin should be in order to achieve the test objectives. In order to confirm that these requirements have been met, it would normally be necessary to re-test following incorporation of all design and process changes, to validate their effectiveness. If this is not possible, it will be necessary to assess the risks involved where a particular design or process change has not been validated.

Page 11 of 20

Reliability Specification & Test RST

+100% +80% +60% +40% +20% =100% Stress Level

Failure Mode 1 safety margin Failure Mode 2 fails at minimum life Failure Mode 3 fails before minimum life

Area bounded by 100% stress and minimum life Component Minimum Life Figure 7. Test Result Analysis

When a test has reached a point where sufficient confidence has been achieved via an acceptable stress margin, or failures have not occurred by application of a predetermined maximum stress level within a specified period of time, it is possible to conclude the test. 4.6 Analyse Cause of Failure

The mode and root cause of every failure must be determined rapidly using techniques such as Fault Tree Analysis (FTA) , Cause and Effect Analysis, and Design of Experiments (DOE). All failures should be documented in a First-hand Information Report (FIR) and prioritised for investigation. It is recommended that the standard Problem Solving Method is used during this stage to increase the rigor for solution development. The failures should also be cross-referenced to the FMEA, with particular attention being paid to failures identified in testing that were not previously predicted in the FMEA. If the failure was due to a weakness, which would be present in all examples, then a design improvement is needed. If the weakness is only going to be present in some examples, the process used to make the component is too variable and should be improved. However, it may be necessary to make a design improvement as well as improving the process if the design is not robust to process variations. The test programme should continue until all of the test objectives have been achieved. When the identified failure modes have been closed off with appropriate solutions this should be documented in an Action Taken Report (ATR). 4.7 Does the Specification need to be Modified?

Having made the decision that an improvement to either the design and/or the manufacturing process is necessary, work will be carried out to make the design or process robust enough to achieve the component reliability specification. The decision does the specification need to be modified? is required in case the component specification cannot be met even after re-design or process changes, or substantial change is required. 5.0 RST Process linked to the NPD Gateways

Page 12 of 20

Reliability Specification & Test RST

The RST process is conducted in phases matching the NPD process in order to facilitate management of the entire reliability specification and test process. It is particularly important in that RST by its very nature links a number of key processes appearing in the NPI documentation; APQP, FMEA, Product Definition, Road Load and Customer Usage. An overview of the RST process mapped onto the DR1, 2 and 3 Gateways follows and should be used in planning the Reliability Management Plan called for at DR2 and in executing the associated procedures contained in this Handbook.

Page 13 of 20

Reliability Specification & Test RST

Establish Vehicle Minimum/Maximum. Life

Cost of Repair Technical Targets and Functional Specification Time Off Road Severity Activity Rating

RCP's

RCI's

Calculate System Minimum Lifes

E' Values

DR1
Outline PFMEA Conduct System FMEA

RPN's - Prioritise See Appendix 3 Reliability Management Plan

DR2
Component FMEA Cascade Road Load and Usage Data

Redesign for Robustness

Test Cycles @ Load and Environment

Determine Test Order and Level

System/Component Reliability Test Plan

DR3
Reliability Tests See Figure 5

Page 14 of 20

Reliability Specification & Test RST

Appendix 1

APQP Timing Chart

Appendix 2

Page 15 of 20

Reliability Specification & Test RST

Process for Establishing the Minimum Life

THE MINMUM LIFE ALOGORITHM

E = A [ B + C + (0.5D) ]
A = Severity Rating Scale (1 to 10) B = Cost of Repair Rating Scale (1 3 5) C = Time Off Road Rating Scale (1 3 5) D = Activity Rating Scale (1 3 5) E = Minimum Life (1 to15 is short 16 to 35 is medium over 35 is long)

NB: Coverting short, medium, long to time/mileage is project team task

A. Severity Rating Scale

Immediate loss of control likely to resulting in safety hazards 9 or 10 Immediate reduction in control but no safety hazards Extreme customer irritation or loss of use of the vehicle Significant customer irritation with use of vehicle Minor problems not noticed by the majority of customers 7 or 8 5 or 6 3 or 4 1 or 2

NOTE:

For severity ratings of 8 to 10 the minimum life of the component or system is the life of the vehicle. Therefore there is no need to use the minimum life algorithm.

B. Cost of Repair Rating Scale

Low up to Rs 500 Medium between Rs 500 and Rs 3000 High above Rs 3000 1 3 5 Project team decision as to required levesl

C. Time Off Road Rating Scale

Page 16 of 20

Reliability Specification & Test RST

Short up to one day (not overnight) Medium between one and three days Long over three days

1 3 5

Project team decision as to required levels

D. Activity Rating Scale LOAD

Heavy/cyclic Moderate Light/none Constant 1 1 3 Intermittent 1 3 5

PERCEIVED USAGE
Rare 3 3 5 Non/static 3 5 5

E. Minimum Life Rating Scale Minimum Life Rating 1 to 15 16 to 35 36 and above Minimum Life Short Medium Long

WORKED EXAMPLE OF CALCULATING MINIMUM LIFE Example 1

Customer concern A. Severity of problem the cars handbrake wont hold the car on a hill rated at 10 due to immediate loss of control likely to resulting in safety hazards

The minimum life for this case is the life of the vehicle because the severity rating is 10, which means that the problem must never be allowed to happen catastrophically during customer usage. Therefore, the engineer must ensure that the problem cannot occur or that there is sufficient advance warning of the problem occurring to allow the customer to have the car repaired.

Example 2
Customer concern A. Severity rating the handbrake grip comes off the lever rated at 4 due to significant customer irritation where the cause is likely to be the handbrake grip is faulty or was not fitted correctly during manufacturing

B. Cost of repair rating C. Time off road rating

rated at 1 as the likely repair is to replace the handbrake grip at a cost of Rs 400 rated at 1 as handbrake grip replacement takes

Page 17 of 20

Reliability Specification & Test RST

D. Activity rating scale

one hour at the dealer so the customer is without the car for up to one day rated at 5 as a typical non-technical customer is likely to perceive the grip as being used intermittently and under little or no load A [ B + C + (0.5 D) ] = E

E. Minimum life rating

4 [ 1 + 1 + (0.5*5) ] = 18 Therefore the minimum life is medium If we assume that the handbrake is from a vehicle in the sub-compact market segment and that the warranty period is three years the short life is four years (warranty period plus one), the medium life is seven years, and the design life is ten years. The analysis of the grip design should reveal the failure modes and causes that result in the grip coming off. Assume that the analysis has shown the critical failure mode is the glue bonding breaking down between the grip and lever caused by fatiguing of the bond due to the lever being pulled up and pushed down, plus the aging of the glue due to atmospheric exposure. Therefore the specification should include a time element to cover the aging plus a number of operations element to cover the fatiguing. The number of typical customer operations can be estimated by considering a typical drive cycle (city drive cycle in this case) and hence the number of operations per day, week, year, and over the seven years minimum life.

Appendix 3

FMEA Test Process

STEP 1 Determine failure modes and critical causes

Page 18 of 20

Reliability Specification & Test RST

FMEA FAILURE MODE ------------------------------------------------------CRITCAL CAUSES ----------------------------------------------------------------

STEP 2

Determine linkages of failure modes and causes

FAILURE MODE/STRESS MATRIX

LOADS/STRESSES Environment Cyclic load Humidity Road inputs Corrosion Etc.

FAILURE MODES FM1 FM2 FM3 FM4 FM5 FM6

STEP 3

Determine optimum test order

1. Corrosion test 2. Cyclic load test 3. Maximum load test

STEP 4

Determine level of testing

FAILURE MODES FM1 FM2 FM3 FM4 FM5 FM6

TEST LEVEL Component Sub-assembly Assembly System

Page 19 of 20

Reliability Specification & Test RST

Vehicle

STEP 5

Set Test Stress Levels

Determine stress levels from data analysis and a review of current practice

Page 20 of 20