You are on page 1of 9

Software Testing

ITECH7409

Assignment 1
Research on Software Testing and Standards
(Individual)

Submitted by:
Research on Software Testing and
Table of Contents

Acknowledgments ......................................................................................................................................... 3
Introduction .................................................................................................................................................... 4
Testing Standards used in the Research Study ............................................................................................. 4
Universities and Institutes Involved in the Study ............................................................................................ 6
Key Terms for Understanding Standards....................................................................................................... 6
Results of the Research Study ...................................................................................................................... 7
Conclusion ..................................................................................................................................................... 7
References .................................................................................................................................................... 9
Acknowledgments
Introduction

Different software testing techniques were introduced in the past so as to determine whether this
can be accepted and adequate based on some standards followed by the users. With this, many
researches came into the field of study so as to evaluate the quality of these testing techniques, comment
on them, and then give suggestions on how to enhance the performance of the said technique based on
the necessary standards.

This paperwork contains discussions and deep understanding regarding a research which has
evaluated a certain software testing technique, whether it is appropriate, reliable, or reasonable, so to be
able to measure the quality of a professional industry standard software. The paper used for this study is
entitled as "Problems and Strategies for Software Component Testing Standards", by B. A. Wichmann and
M. G. Cox (1992). As a brief summary, this paper assesses the currently available standards, mainly in the
component testing area and advocates that the British Computer Society proto-standard should be taken
as a basis for a formal standard. The paper is intended for those concerned with software quality. The
objective of this research is to quantify or assess the quality of the software under test, particularly in an
objective manner which can therefore be agreed by both a supplier and customer. The main factor
considered in evaluation and assessment in this paper is the safety of the software.

Above all testing techniques, the dynamics testing was preferred in this study. For common
knowledge, Dynamic Testing checks for functional behavior of software system, memory/cpu usage and
overall performance of the system. The main objective of this testing is to confirm that the software product
works in conformance with the standard requirements. This testing is also called an Execution technique or
validation testing.

Testing Standards used in the Research Study

The standards used in this research are the following, with the corresponding copyright holders
and the scope and intent of that standard:

BS 5887. This Standard (BSI, 1988) takes the form of guidance material for functional testing of
software, i.e. black-box testing of a complete system. There are no mandatory requirements and therefore
the document has little relevance to this study, which is concerned with the quality implications of a specific
level of test.

MOD 00-55. The U.K. Interim Defence Standard (MOD, 1991) is highly prescriptive on dynamic
testing. The essential requirements are as follows. Test coverage monitor to be used. All statements, and
all conditional branches with both outcomes, to be executed during testing. Loops to be executed with 0, 1
and many iterations. Results to be compared with executable prototype. Module tests results to be
computed in advance from design information.

IEEE. There are two ANSI/IEEE standards on testing (ANSI, 1983; 1987) (and several other
standards which refer to testing in quality management and quality assurance). These two standards are as
follows. (1) ANSI/IEEE Std 829:1983, Standard for Software Test Documentation. It has little direct
relevance to this study since it only handles the documentation rather than the content and nature of the
tests themselves. (2) ANSI/IEEE Std 1008:1987, Standard for Software Unit Testing. The standard could
be considered as a ‘guideline’ rather than a true standard.

JM178B (draft). This document is the current revision of the general avionics standard for safety-
critical software used internationally by both the industry and the certification bodies (RCTA, 1993). The
document has an objective that statements of conditions (both ways) should be executed during testing. is
an interesting additional requirement to ensure that the testing based upon the source code is not defective
in that the same structural testing based upon the object code would require more tests.

IEC/WGB. This International Electrotechnical Commission (IEC) proposed standard (IEC, 1989)
recommends but does not require boundary value analysis and path coverage. The term ‘path coverage’ is
rather unfortunate, since on all but trivial examples, complete coverage of paths is impossible. Hence the
possibility of executing all statements, which is often feasible but difficult, is not considered.

IEC/Nuclear. This international standard for nuclear safety software (IEC, 1986) has an annex
concerned with software testing. This covers both systematic and random testing.

ITSEC. The current version of the Information Technology Security Evaluation Criteria (CESG,
1991) does not specify actual testing methods (this is under development). However, the higher levels of
conformance do require the provision of information that is necessary for independent white-box testing.
Universities and Institutes Involved in the Study

There are also universities who gave contribution for the outcome of this research. The Institute of
Software Engineering (in Northern Ireland) reported that only about one quarter of companies use
regression testing on a routine basis (Thompson, 1991). Similar, rather less quantified experience has been
reported to the author by the Centre for Software Maintenance at Durham University (U.K). They also
state that since testing is at the end of the life-cycle, there is a tendency for the amount of testing to be
reduced to allow the project to complete within budget.

Key Terms for Understanding Standards

The following are the key terms and understanding needed so as to understand and apply the standards

Software testing. It is an investigation conducted to provide stakeholders with information about


the quality of the software product or service under test.[1] Software testing can also provide an objective,
independent view of the software to allow the business to appreciate and understand the risks of software
implementation.

Requirement It is a broad concept that could speak to any necessary (or sometimes desired)
function, attribute, capability, characteristic, or quality of a system for it to have value and utility to a
customer, organization, internal user, or other stakeholder.

Quality a measure of excellence or a state of being free from defects, deficiencies and significant
variations. ISO 8402-1986 standard defines quality as "the totality of features and characteristics of a
product or service that bears its ability to satisfy stated or implied needs."

Prototype is an early sample, model, or release of a product built to test a concept or process or to
act as a thing to be replicated or learned from. A prototype is generally used to evaluate a new design to
enhance precision by system analysts and users.[2] Prototyping serves to provide specifications for a real,
working system rather than a theoretical one.

Safety-critical From a software perspective, developing safety-critical systems in the numbers


required and with adequate dependability is going to require significant advances in areas such as
specification, architecture, verification and the software process. The very visible problems that have arisen
in the area of information system security suggests that security is a major challenge too.
Path coverage Path coverage refers to designing test cases such that all linearly independent
paths in the program are executed at least once. A linearly independent path can be defined in terms of
what’s called a control flow graph of an application.

Results of the Research Study

The results and findings of this research is divided into the following points, and these points are
the suggestions for the total enhancement of the standards used for this research. Some are present in
each standard already, but some are lacking. Thus, this summarizes the suggested optimal requirement for
a quality testing of a software.

(1) It should be objective. The subjective nature of informal testing undertaken by most suppliers is
such that the phrase ‘it has been tested’ is meaningless. Repeatable tests with clearly defined results must
underlie the testing process.

(2) It should be enough for users. Any formalized system must provide enough guarantees that
even the minimal level of testing ensures a significant evidence of software quality.

(3) It should be practical for suppliers. It is easy to specify a level of test that is totally uneconomic.
A simple and measurable statement of what to do, why and when (in the software lifecycle) is needed.

(4) It should match the perceived risks. There is a wide range of software, and any testing
approach that only addresses the most critical software will have little impact on the majority of software
developers.

(5) It should have conceptual simplicity. Effective testing requires resources. In consequence, all
parties must have a clear understanding of the implications of the level of testing envisaged, in both costs
and benefits.

Conclusion

As a conclusion for this paperwork, the relevance of standards to software testing and other
measuring techniques are stressed. Standards form the fundamental building blocks for product
development by establishing consistent protocols that can be universally understood and adopted. This
helps fuel compatibility and interoperability and simplifies product development, and speeds time-to-market.
Standards also make it easier to understand and compare competing products. As standards are globally
adopted and applied in many markets, they also fuel international trade.

It is only through the use of standards that the requirements of interconnectivity and interoperability
can be assured. It is only through the application of standards that the credibility of new products and new
markets can be verified. In summary standards fuel the development and implementation of technologies
that influence and transform the way we live, work and communicate.
References

ANSI (1983) ANSUIEEE Std 829: 1983, Standard for Software Test Documentation.
ANSI (1987) ANSI/IEEE Std 1008:1987, Standard for Software Unit Testing.
BSI (1988) BS 5887: Code of Practice for Testing of Computer-based Systems, British Standards
Institution, London, U. K.
CESG (1991) ‘Information technology security evaluation criteria: provisional harmonised criteria, Version
1.2. (U.K. contact point: CESG Room 2/0805, Fiddlers Green Lane, Cheltenham, Glos, GL52 5AJ.)
IEC (1986) IEC 880:86: Software for Computers in the Safety Systems of Nuclear Power Stations.
IEC (1989) IEC/SC65AIWG9/45: Software for Computers in the Application of Industrial Safety-
Thompson, K. (1991) ‘A method for assessing organisational software development capability’, EuroCASE
111 Conference
MOD (1991) Interim Defence Standard 00-55: The Procurement of Safety Critical Software in Defence
Equipment (Part 1: Requirements; Part 2: Guidance), Ministry of Defence.
RCTA (1993) ‘Software considerations in airborne systems and equipment certification’ (DO-178-B),
Requirements and Technical Concepts for Aviation. 1140 Connecticut Avenue, N.W., Suite 1020
Washington, DC 20036, U.S.A.
Wichmann, B. A. and Cox, M.G. (eds), (1992) Problems and Strategies for Software Component Testing
Standards, Wiley.