You are on page 1of 24

System test

,
Evaluation, and
Validation
By:
Ocampo,
Abbygail
Zabat, Jenilyn

System test, Evaluation and
Validation
System test, evaluation, and validation activities should be established
during the conceptual design phase of the life cycle, concurrently with
the definition of the overall system design requirements. From that
point on, The test and evaluation continues by testing of individual
components, the testing of various system elements and major
subsystems and then by the testing of overall system as an integrated
entity.
The objective is to adopt a “progressive” approach that will lend it self to
continuous implementation and improvement as the system design
and development process evolves.
Test and evaluation activities in this chapter can be aligned initially with
the design activities describe in chapter3-5 and then extended through
production/construction and system utilization.

The purpose of this chapter is to: • to present an integrated approach for system • • test and evaluation to facilitate the necessary validation of the proposed system configuration and to provide assurance that will indeed meet the customer requirements .

System Requirements and Evaluation Relationship .

Stages of System Evaluation During Life Cycle .

2. potential interferences.Categories of System test and Evaluation • 6. for example. From this. the development of system through computer graphics can illustrate the relationships among various system elements and accesses for maintenance and support. .1 Analytical and Simulation Evaluation The utilization of analytical models. CAD methods. space utilization conflicts can be detected. and various combinations of these can be effective in the verification of certain design relationships before system is finalized.

breadboards. representing either an entire system or a designated system component. engineering software. In the development of software. engineering models. usually operate functionally (electrically and mechanically) but do not by any means represent production equipment.Categories of System test and Evaluation • Type 1 Testing During the early phase of detail design. the application of rapid prototyping is sometimes used to verify design adequacy. These models. . and service test models are build with the intent of verifying certain performance and physical design characteristics. bench-test models.

software. A test program may constitute a series of individual tests tailored to the need.Categories of System test and Evaluation • Type 2 Testing Formal tests and demonstrations are accomplished during the latter part of detail design when preproduction prototype equipment. but it is not necessarily fully qualified at this point in time. and similar to production equipment (that which will be deployed for operational use). Such a program might include the following: .

. fit.Type 2 test. Environmental qualification – temperature cycling. dust and sand. For instance. whether the airplane will perform its intended mission successfully. it is necessary to verify form. and so on. 2. • • 1. fungus. interchangeability.. and during transportation and handling function. Also. spray. maintenance. acoustic noise. and electromagnetic interference tests are conducted. product safety. wind. explosion proofing. pollution emission. tests are designed to determine whether the electric motor will provide the necessary output. Performance tests are accomplished to verify individual system performance characteristics. shock and vibration. and other comparable features. salt. humidity. These factors are oriented to what the various system elements will be subjected to during operation.

fatigue. Structural tests are conducted to determine material characteristics relative to stress. special tests are often designed to measure component life. Maintainability demonstration – tests are conducted on one or more system elements to assess the values for mean active maintenance time (M). and to determine models of failure. to evaluate degradation. mean preventive maintenance time (Mpt).Type 2 test. strain. and so on . 4. 3. 5. torsion and general decomposition. Also. bending. maintenance labor-hours per operating hour (MLH/OH). Reliability qualification are accomplished on one or more system elements to determine MTBF.. mean corrective maintenance (Mct).

test and support equipment.Type 2 test. Support equipment compatibility tests are often accomplished to verify compatibility among the prime equipment. Technical data verification – the verification of operational and maintenance procedures is accomplished. .. 9. 6. the personnel quantities and skill levels required and training needs. Software verification – the verification of operational and maintenance software is accomplished. Personnel test and evaluation are often accomplished to verify the relationships between people and equipment. and transportation and handling equipment 7. both operator and maintenance tasks are evaluated. 8.

are accomplished at a designated field test site by user personnel. . operation.Categories of System test and Evaluation • Type 3 Testing Formal tests and demonstrations. started after initial system qualification and prior to the completion of production.

Categories of System test and Evaluation • Type 4 Testing During the operational-use phase. or it might be feasible to evaluate several alterative support policies to see whether system operational availability can be improved . formal tests are sometimes conducted to gain further insight in a specific area. It may be desirable to vary the mission profile or the system utilization rate to determine the impact on total system effectiveness.

3. . The definition of organization. including anticipated test output for each individual test and integrated where possible. The definition of test conditions and maintenance and logistic resource requirements.Planning for Testing and Evaluation 1. administration and control responsibilities. 2. The definition and schedule of all tests requirements.

training of testing. 7. 5.Planning for Testing and Evaluation 4. A description of condition and provisions for a retest phase. 6. A description of the formal test phase. The identification of test documentation . A description of the test preparation phase for each type of testing method.

Evaluation of Test Requirements .

Acquisition of support equipment. .Preparation for system test and evaluation • • • • • • • The selection of the items to be tested. Test supply support. Establishment of test procedures. Preparation of test facilities and resources. Selection and training of test personnel. Test site selection.

Test performance and reporting • This requires operating and supporting the system in a prescribed manner as defined in the system test and evaluation plan. .

are the initial specified requirements being met? . what is the true effectiveness of the maintenance and logistic support capability? 3. what is the true performance and effectiveness of the system? 2.Test performance and reporting • The following questions will be answered: 1.

logistic support capability. It provides historical data that are applicable in the design and development of new systems having a similar function and nature. .Test data requirements 1. operation. maintenance. 2. It provides ongoing data that are analyzed to evaluate and assess the performance. and so on. effectiveness.

. 2. Maintenance data cover each event involving scheduled and unscheduled maintenance.Development of a data subsystem 1. Success data constitute information covering system operation and use on a day-to-day basis.

Test reporting 1. It should describe all test conditions. . incorporated system modifications during the test. test data. 3. The final effort in the evaluation process. It should reference the initial system evaluation planning document. and the results of data analysis. 2.

System Evaluation and Corrective Action Loop .

Summary .