Conformance Testing

Lynne S. Rosenthal Software Diagnostic and Conformance Testing Division
(301)975-3353 lynne.rosenthal@nist.gov

Outline
 Software

Standards and Testing  Conformity assessment  Conformance Testing
– what is it – why bother – how it works

Software Standards
 Promote

portability & interoperability  Success depends on:
– Efficiency of standard development process – Quality of resulting standard – Ability to ensure implementations conform to the standard

Software Testing
 Testing

- process of trying to find errors by means of experimentation  Testing of realistic system not exhaustive
– can show presence of errors not their absence
 Testing

is either

– structural testing (white box) – functional testing (black box) • conformance testing

Conformance Testing
 Determines

which areas of the specification* are implemented correctly.
– tests capture the technical description of the specification – testing is bound in scope by the specification – internal structure of product-under-test not accessible

*publicly available standard

Why: conformity assessment
 ISO

Guide 2: “any activity concerned with determining directly or indirectly that relevant requirements are fulfilled.” between buyer and seller

 communication

– buyers increased confidence – sellers substantiate claims

Why:
 Improves

quality of software  Objective method for evaluating products
– ensures presence of required characteristics
 Promote

portability and interoperability

How: building tests
 Falsification

Testing

– find errors by means of experimentation – show presence of errors not their absence – prove non-conformance
 Tests

based on:

– required functionality – logical errors by misunderstanding requirements – errors from boundary conditions and divergence – common programming errors

Requirements

Software

Conformance Tests
Test programs

When:
 Develop

in parallel with specification

– feedback to specification for clarification, correction, etc.
 Develop

in partnership with product developers

Summary: Conformance tests
 Determine

if a product has required product confidence between buyer and seller

characteristics
 Increases  Communication  Improve

quality of software

Sign up to vote on this title
UsefulNot useful