This action might not be possible to undo. Are you sure you want to continue?
What s Software test?
y Any software product, no matter how
accomplished by today s testing technologies, has bugs. y Software testing is the process of detecting and removing the bugs from the software product. y The goal of software testing is ensuring of software product quality according to the customer requirements.
Some typical software testing questions:
How long will it take to test software? How much will it cost to test? What kind of bugs were found? How many of the bugs that were found were fixed? Will the product be ready on time? How much is the risk of release?
y In ideal a metric must have these characteristic: y simple y Applicable y Easily collectable y Valid y Robust 4 .Test Metrics y Metrics are ways of answering questions. y There are numerous metric for software testing.
Software Metrics 5 .
Test Metrics y Test metrics classifies in two category: y Product Metrics: Metrics that are related to test results or quality of software under test y Process Metrics: Metrics that used for evaluation of testing process effectiveness 6 .
Code which tolerates faults will not. by definition. y An approach to determining testability involving fault injection in three stages: y y y propagation infection execution 7 . exhibit them. y Software can be temporarily rendered more testable by including assertions in it.Testability y Testability can be considered the inverse of fault tolerance.
Bug (Fault) Density y Each system has two characteristics: y the number of bugs y and the number of non-commented lines of source code (KLOC) y Bug Density can be used to predict release readiness 8 .
weights assign according to user viewpoint y N.Weighted Fault Density y Bug density can be calculated by bug severity. W3 . Fault Density (Weighted) = W1 S / N + W2 A / N + W3 M/ N y W1. number of severe faults y A. W2. number of average fault y M. number of faults y . number of minor faults 9 .
Many ranking schemes exist for defining severity. 10 .Severity y Severity is a fundamental measure of a bug or a failure.
11 . y The graph presented here can be used successfully to allocate test and development resources toward the end of the test effort so that the product could be released on time.Bugdensities should be monitored throughout the test Density per Unit y Bug effort.
and low-level design documents used per KLOC 12 . y The Number of bugs per KLOC= y f1 = the number of bug reports required to change the design and code during the coding phase per KLOC y f2 = the average number of years experience of programming of all programmers involved y f3 = the number of pages of new and modified high.Bug Prediction KLOC Estimate y This model can be applied only when coding is finished.
of test hours since the last problem was observed y f = expected number of field failures y s = number of test failures discovered to far 13 . y a = no.Zero-Failure Approach to Release Readiness Estimation y It is based on some period being defined during which no further failures must be found if some level of reliability is to be achieved.
y cost of the testers' salaries. systems. y cost to fix. 14 . y cost in lost revenues.Bug Cost y The bug cost usually includes the y cost of bug analysis. y Cost of equipment. and other tools. software.
Bug Find Rate y This is a most useful derived metric both for measuring the cost of testing and for assessing the stability of the system. y It can give a good indication of the stability of the system being tested. 15 .
Coverage y There are some coverage metric: y Code coverage y Branch coverage y Path coverage y Test coverage y Data flow coverage 16 .
CPU cycles) and assumes: y Constant execution load y Measurement against some error threshold 17 .g.Mean Time to Failure y This is measured usually against a clock (e.
Availability y Availability is the probability that the system will still be operating to requirements at a given time. 18 . y One way of determining a system s availability is to model it as a Markov chain.
It does not specify the required set of test documents. IEEE Standard for Software Test Documentation y The purpose of this standard is to describe a set of basic software test documents. y It is assumed that the required set of test documents will be specified when the standard is applied.IEEE Std 829. 19 . y This standard specifies the form and content of individual test documents.
y Increase visibility of each phase of the testing process.IEEE Std 829 y Benefits: y Facilitate communication by providing a common frame of reference y Provide a baseline for the evaluation of current test documentation practices. 20 . y The use of these documents significantly increases the manageability of testing.
A test log. approach.y The documents outlined in this standard cover test planning. A test summary report 21 . A test case specification. A test incident report. resources. and test reporting. y Test Planning Documents and schedule of the testing activities. A test procedure specification y Test Reporting y A test item transmittal report. y Test Specification Documents y A test design specification. test specification. y The test plan prescribes the scope.
y ISO/IEC 9126 (1991) has been replaced by two related multipart standards: ISO/IEC 9126 (Software product quality) and ISO/IEC 14598 (Software product evaluation) 23 . Software Engineering Product Qualityy ISO/IEC 9126 (1991): Software product evaluation - Quality characteristics and guidelines for their use.ISO/IEC 9126(1991). which was developed to support these needs.
under the general title oftware engineering y Part 1: Quality model y Part 2: External metrics y Part 3: Internal metrics y Part 4: Quality in use metrics Product quality: 24 .ISO/IEC 9126 y ISO/IEC 9126 consists of the following parts.
y First part of ISO/IEC 9126 describes a two-part model for software product quality: internal quality and external quality b) quality in use. a) 25 .
Quality model for external and internal quality 26 .
productivity.Quality in Use y The capability of the software product to enable specified users to achieve specified goals with effectiveness. 27 . safety and satisfaction in specified contexts of use.
Relationship between ISO/IEC 9126 and ISO/IEC 14598 standards 28 .
29 .IEEE P1671. y This information includes y test data y resource data y diagnostic data y historic data. Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipment and Test Information via XML y ATML defines a standard exchange medium for sharing information between components of automatic test systems.
sharing. 30 . Supports modular software architectures based upon a framework that supports reusable software products. Facilitate TPS portability and interoperability. .ATML Objective y ATML is intended to accomplish the following objectives: y Facilitate the communication. and use of test software and test software development tools. and reuse of product design y y y y and test information for the purpose of testing the product. integration. Facilitate the development.
ATML Components are segmented into: y ATML Component Standards (in the form of formal IEEE Standards). the ATML framework is defined in the form of ATML Components. The ATML Components define the domain-orientated information. y Reference to ATML Instance Documents 31 . y XML Schemas.IEEE P1671 y As previously stated.
Software Engineering Product Qualityy IEEE P1671.Conclusion y Software testing metrics y Product metrics y Process metrics y Software testing standards y IEEE 829: Software Test Documentation y ISO/IEC 9126(1991). ATML for Exchanging Automatic Test Equipment and Test Information via XML 33 .
Is your software ready for release? IEEE Software. Manage Software Testing. [IEEE90] IEEE (1990). Defines the content and format of documents that cover the testing process. Software Testing Fundamentals: Methods and Metrics.Draft Trial-Use Standard for Automatic Test Markup Language (ATML) for Exchanging Automatic Test Equipmqnt and Test Information via XML. John Wiley & Sons. New York: IEEE. ISO Genève. 2008. [ISO/IEC 9126] ISO/IEC standard 9126 Software engineering product quality Part 1: quality model. July 1989. Hutcheson.References [Bre89] Ralph Brettschneider(Motorola). 2003. [Far08] Peter Farrell-Vinay. IEEE Standard Computer Dictionary: A Compilation of IEEE Standard Computer Glossaries. [Hut03] Marnie L. Auerbach Publication. [IEEE 829] IEEE Std 829-1998 Software Test Documentation. [IEEE P1671] IEEE Std P1671-2006 Archived Approved Draft . 2001 34 . ISBN 1559370793.
35 . ISO Genève. 2001 [Mic02] James B. Kamayachi. [Nag05] Nachiappan Nagappan. [Tak89] M. A Software Testing and Reliability Early Warning (STREW) Metric Suite. Bossuyt. IEEE Transactions on Software Engineering. 2005. Takahashi and Y. and Byron B. Snyder. Bernard J. January 1989. Michael. A dissertation submitted to the graduate faculty of North Carolina State University.References [ISO/IEC 9126] ISO/IEC standard 9126 Software engineering product quality Part 1: quality model. An empirical study of a model for program error prediction. Proceedings of the 13th International Symposium on Software Reliability Engineering (ISSRE 02). Metrics f or measuring the effectiveness of software-testing tools.
Question? 36 .
This action might not be possible to undo. Are you sure you want to continue?
We've moved you to where you read on your other device.
Get the full title to continue reading from where you left off, or restart the preview.