5/8/2015

REKAYASA PERANGKAT LUNAK

(IK370)
Software Testing Techniques -1

Asep Wahyudin, S.Kom, MT.

Ilmu Komputer
FPMIFA - Universitas Pendidikan Indonesia

What is Software Testing

(The Historical definition of testing)

1. Establishing confidence that program does what it is supposed to do (Hetzel, 1973)

2. The process of executing a program or system with the intent of finding errors
(Myers,1979)
3. Detecting specification errors and deviation from the specification
4. Any activity aimed at evaluating an attribute or capability of a program or system
(Hetzel, 1983)
5. The measurement of software quality (Hetzel, 1983)
6. The process of evaluating a program or system
7. Verifying that a system satisfies its specified requirements or identifying
differences between expected and actual resuls
8. Confirming that a program performs its intended function properly

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

2
IK370

1
 5/8/2015

What is Software Testing

(The Historical definition of testing)

IEEE / ANSI
1. The process of operating a system or component under specified conditions,
observing or recording the results, and making an evaluation of some aspect
of the system/component

2. The process of analyzing a SW item to detect the difference between existing

and required conditions (that is, bugs) and to evaluate the features of the SW
items.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

3
IK370

Software Testing

Testing is the process of exercising a

program with the specific intent of finding
errors prior to delivery to the end user

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

4
IK370

2
 5/8/2015

Software Testability

 Operability it operates cleanly

 Observability the results of each test case are readily
observed
 Controllability the degree to which testing can be automated
and optimized
 Decomposabilitytesting can be targeted
 Simplicity reduce complex architecture and logic to
simplify tests
 Stability few changes are requested during testing
 Understandabilityof the design

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

5
IK370

Testing Show

errors
requirements conformance

performance

an indication of quality

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

6
IK370

3
 5/8/2015

Who Tests the Software

Developer
Understands the system but, will test "gently“and, is driven by "delivery"

Independent Tester
Must learn about the system, but, will attempt to break it and, is driven by quality

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

7
IK370

The Software Testing Associated

SW Customer
The Group that contracts for SW to be developed
SW User
The Group that will use the SW
SW Developer
The Group that develops the SW
SW Tester
The Group that performs the check function on the SW
Information Services Management
The group with the responsibility for fulfilling the Information Service Mission
Senior Organization Management
The executive
Auditor
The group having the responsibility to evaluate the effectiveness, efficiency and the
adequacy of control in information services area.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

8
IK370

4
 5/8/2015

Who Tests the Software

Testers hunt errors

a good test is one that has a good probability of detecting an as yet undiscovered error, and a successful test
is one that detects undiscovered error.
Focus on showing the presence of errors.

Testers are destructive - but creatively so

Testing need imagination, persistence and a strong sense of mission to systematically locate the weaknesses
and demonstrate its failure.

Testers pursue errors, not people

Errors area in the product, not in the person who made the mistake.
Developer should understand that testers are not “againts” them, but are helping them.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

9
IK370

How they detecting the errors

 By examining the internal structure and design

 By examining the functional user interface

 By examining the design objectives

 By examining the users’ requirements

 By executing code

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

10
IK370

5
 5/8/2015

What’s Defect

Defect is a variance from a desired product attribute

– Defect from product specification
– Variance from customer/user expectation

Categories of defects
– Wrong - The specification have been implemented incorrectly (variance from user)
– Missing - A specified requirement is not built into product (variance from product specification)
– Extra - A requirement incorporated into the product that was not specified (variance from product specification)

Defect vs Failure
– Defect is incorporated in software system (it can be found within the SW, manuals or documentation) and a flaw
– Defect that causes an error in operation is called a failure

Defect will turn into failure, the failure will damage the organization

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

11
IK370

Errors, Mistake...

Some terms
– Mistake: a human action that produces an incorrect result
– Fault : An incorrect step, process, or data definition in a computer program.
The outgrowth of the mistake (potentially leads to failure)
– Failure : An incorrect result. The result (manifestation) of the fault
(e.g., a crash)
– Error : The amount by which the result is incorrect

Mistakes are what people make

Failure are the manifestation of these fault

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

12
IK370

6
 5/8/2015

Categories of Software Errors

User Interface Error

– Functionality
– Communication
– Command Structure
– Missing Command
– performance
– Output
Error Handling Race Conditions
Boundary-Related Errors Load Conditions
Calculation Errors Hardware
Initial and Later States Source and Version Control
Control Flow Errors Documentation
Errors in Handling/Interpreting data Testing Errors

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

13
IK370

Common Computer Problem

Software Problem
– Incomplete Design or Erroneous decision-making criteria
• actions have been incorrect
• inappropriate decision making criteria
– Fail to meet customer requirement
• logic error or programming error
– Ommitting needed edit checks for completeness of output data

• Data Problems
– Data Integrity
• Incomplete Data
• Incorrect Data
• Inconsistency Data
• Obsolete Data

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

14
IK370

7
 5/8/2015

The Structured Approach To Testing

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

15
IK370

Basic Forms of Testing

Full Testing
starts no later than the requirement phase and continues through acceptance testing
Partial Testing
begins any time after functional design has been completed, with less than optimal influence on
requirements and functional design
Endgame Testing
is highly validation oriented, with no influence on requirements or functional design
Audit-Level Testing
is a barebones audit plans,
procedures,
and products for adequacy,
correctness,
and compliance to standards.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

16
IK370

8
 5/8/2015

Black Box Testing

requirements

output

input

events

# Black Box Testing focuses on functional requirements of the software

Tester can derive sets of input conditions that will fully exercise all functional requirements for a program.

# Black Box Testing is not an alternative to white box testing, rather it is a complementary approach that
is likely to uncover a different class of errors than white box methods.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

17
IK370

Black Box Testing

# Attempts to find:
incorrect or missing functions, interface errors, errors in data structures or
external database access, performance errors, initialization and termination
errors.

# Tests are designed to answer the following question

– How is functionally validity tested?
– What classes of input will make good test cases?
– Is the system particularly sensitive to certain input values?
– How are the boundaries of a data class isolated
– What data rates and data volume can the system tolerate?
– What effect will specific combinations of data have on system operation?

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

18
IK370

9
 5/8/2015

Black Box Testing

 Equivalence Partitioning
 Boundary Value Analysis/Limit Testing
 Comparison Testing
 Sample Testing
 Robustness Testing
 Behavior Testing
 Requirement Testing
 Performance Testing
 Endurance Testing
 Cause-Effect Relationship Testing

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

19
IK370

Black Box Testing:

Equivalence Partitioning

* Equivalence classes can be defined by:

• If an input condition specifies a range or a specific value, one valid and two invalid equivalence classes
defined.
• If an input condition specifies a boolean or a member of a set, one valid and one invalid equivalence classes
defined.
* Test cases for each input domain data item developed and executed.

Example:
• Requirements of the subprogram to be tested (initial&final state)
– The subprogram takes an integer input in the range [-100,100]
– The output of the subprogram is the sign of the input value (the value 0 is considered positive)
• Two valid equivalence class
– The input range [-100,-1] which contains the subset of integer, will produce negative sign as an output
– The input range [0,100] will produce positive sign
• Two invalid equivalence class
– The integers strictly less than -100
– The integers strictly more than 100

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

20
IK370

10
 5/8/2015

Black Box Testing:

Boundary Value Analysis/Limit Testing

• Large number of errors tend to occur at boundaries of the input domain.

• BVA leads to selection of test cases that exercise boundary values.
• BVA complements equivalence partitioning. Rather than select any element in an equivalence class,
select those at the ''edge' of the class.

Examples:
– For a range of values bounded by a and b, test (a-1), a, (a+1), (b-1), b, (b+1).
– If input conditions specify a number of values n, test with (n-1), n and (n+1) input values.
– Apply 1 and 2 to output conditions (e.g., generate table of minimum and maximum size).
– If internal program data structures have boundaries (e.g., buffer size, table limits), use input data to
exercise structures on boundaries.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

21
IK370

Black Box Testing:

Comparison Testing

• In some applications the reliability of SW is critical.

Aircraft avionics, nuclear power plant control
• Redundant hardware and software may be used to minimize error.
• For redundant SW, use separate teams to develop independent versions of the software.
• Test each version with same test data to ensure all provide identical output.
• Run all versions in parallel with a real-time comparison of results.
• Even if will only run one version in final system, for some critical applications can develop
independent versions and use comparison testingor back-to-back testing.
• When outputs of versions differ, each is investigated to determine if there is a defect.
• Method does not catch errors in the specification.

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

22
IK370

11
 5/8/2015

Black Box Testing:

Sample Testing

• Sample testing involves selecting a number of values from an equivalence class of input data

• Integrate the values into test cases

• These values may be chosen at constant or variable intervals

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

23
IK370

Black Box Testing:

Robustness Testing

• Data are chosen outside the range defined by the requirements

• The purpose of this test is to prove that no catastrophic event can be produced by the
introduction of an abnormal input data item

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

24
IK370

12
 5/8/2015

Black Box Testing:

Behavior Testing

• A behavior tests is a test for which the result obtained cannot be evaluated after a single
execution of program, but can be evaluated only after a set of linked calls to subprogram
(several calls to a given subprogram, a call to several different subprogram)

– E/g : stack

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

25
IK370

Black Box Testing:

Requirement Testing

• The requirement associated with the software (input/output/function/performance) are

identified during the software specification and design activities.
• Requirement testing involves producing a test case for each requirement
• To facilitate the implementing of such tests, each requirement may be traced to final test case
trough the traceability matrix

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

26
IK370

13
 5/8/2015

Black Box Testing:

Performance Testing

• Performance testing evaluates the computer system’s ability to operate correctly with regard
to the requirement concerning, for

example:
– data flow, memory size, execution time, etc

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

27
IK370

Black Box Testing:

Endurance Testing

• Endurance Testing involves repeating a given test case a certain number of times in order to
evaluate a computer system’s ability to meet requirements.

• For example:
– the regularity in the accuraccy of certain mathematics operations
(floating point, rounding off, etc)

• These requirements are defined during specification or design activities

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

28
IK370

14
 5/8/2015

Black Box Testing:

Cause-Effect Relationship Testing

• This technique supplements equivalence testing by providing ways of determining and selecting
combinations of input data
• It involves representing the input state (cause) and output states (effect) deduced from functional
requirements associated with the computer system in the form of a graph represented by logical relationship
(decision table), to avoid of having defined too many test cases

• The Steps are:

– Break down the requirements into subset within which it will possible to work
– Define cause and effect based on the requirement
– Analyze the requirements to make a logical relationship
– Highlight the graph, the impossibilities of combinations of cause/effect
due to constraint from the requirements
– Convert the graph into a decision table
• Column --> test case
• Row --> cause/effect
– Convert the columns of the decision table into test cases

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

29
IK370

Black Box Testing:

Cause-Effect Relationship Testing

• A CITROEN car will be identified by a letter A, B or C, and have the letter X as the second
character. Messages M1 and M2 must be sent in the event of an error in the first or second
character respectively. If the identifier is correct, it is inserted in the database.
• The the input state (the causes) and the output states (the effect) can be determined:

Input states:
1. 1st character: A
2. 1st character: B
3. 1st character: C
4. 2nd character: X

Output states:
A. database
insertion
B. message M1
C. message M2

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

30
IK370

15
 5/8/2015

Selesai

: Rekayasa Perangkat Lunak: : Asep Wahyudin, S.Kom, MT.- 2398 :

31
IK370

16