ȀȀ

 
ʹͻͳͳͻǦͶ

‹”•–‡†‹–‹‘
ʹͲͳͷǦͳʹǦͲͳ

‘ˆ–™ƒ”‡ȉƒ†ȉ•›s–‡•ȉ‡‰‹‡‡”‹‰Ȅ
‘ˆ–™ƒ”‡–‡•–‹‰Ȅ
ƒ”–Ͷǣ
‡•––‡…Š‹“—‡•
‰±‹‡”‹‡†—Ž‘‰‹…‹‡Ž‡–†‡••›•–°‡•Ȅ••ƒ‹•†—Ž‘‰‹…‹‡ŽȄ
ƒ”–‹‡ͺǣ‡…Š‹“—‡•†ǯ‡••ƒ‹

‡ˆ‡”‡…‡—„‡”
ȀȀʹͻͳͳͻǦͶǣʹͲͳͷȋȌ

̹ȀʹͲͳͷ
̹ʹͲͳͷ

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ȀȀʹͻͳͳͻǦͶǣʹͲͳͷȋȌ

 †‹•…Žƒ‹‡”
Š‹•   ˆ‹Ž‡ ƒ› …‘–ƒ‹ ‡„‡††‡† –›’‡ˆƒ…‡•Ǥ  ƒ……‘”†ƒ…‡ ™‹–Š †‘„‡̵• Ž‹…‡•‹‰ ’‘Ž‹…›ǡ –Š‹• ˆ‹Ž‡ ƒ› „‡ ’”‹–‡† ‘” ˜‹‡™‡† „—–
•ŠƒŽŽ ‘– „‡ ‡†‹–‡† —Ž‡•• –Š‡ –›’‡ˆƒ…‡• ™Š‹…Š ƒ”‡ ‡„‡††‡† ƒ”‡ Ž‹…‡•‡† –‘ ƒ† ‹•–ƒŽŽ‡† ‘ –Š‡ …‘’—–‡” ’‡”ˆ‘”‹‰ –Š‡ ‡†‹–‹‰Ǥ 
†‘™Ž‘ƒ†‹‰ –Š‹• ˆ‹Ž‡ǡ ’ƒ”–‹‡• ƒ……‡’– –Š‡”‡‹ –Š‡ ”‡•’‘•‹„‹Ž‹–› ‘ˆ ‘– ‹ˆ”‹‰‹‰ †‘„‡̵• Ž‹…‡•‹‰ ’‘Ž‹…›Ǥ Š‡  ‡–”ƒŽ ‡…”‡–ƒ”‹ƒ–ǡ–Š‡
‡–”ƒŽˆˆ‹…‡ƒ††‘‘–ƒ……‡’–ƒ›Ž‹ƒ„‹Ž‹–›‹–Š‹•ƒ”‡ƒǤ
†‘„‡‹•ƒ–”ƒ†‡ƒ”‘ˆ†‘„‡›•–‡•…‘”’‘”ƒ–‡†Ǥ
‡–ƒ‹Ž• ‘ˆ –Š‡ •‘ˆ–™ƒ”‡ ’”‘†—…–• —•‡† –‘ …”‡ƒ–‡ –Š‹•   ˆ‹Ž‡ …ƒ „‡ ˆ‘—† ‹ –Š‡
‡‡”ƒŽ ˆ‘ ”‡Žƒ–‹˜‡ –‘ –Š‡ ˆ‹Ž‡Ǣ –Š‡  Ǧ…”‡ƒ–‹‘
’ƒ”ƒ‡–‡”• ™‡”‡ ‘’–‹‹œ‡† ˆ‘” ’”‹–‹‰Ǥ ˜‡”› …ƒ”‡ Šƒ• „‡‡ –ƒ‡ –‘ ‡•—”‡ –Šƒ– –Š‡ ˆ‹Ž‡ ‹• •—‹–ƒ„Ž‡ ˆ‘” —•‡ „›  ‡„‡” „‘†‹‡•ƒ†
‡„‡”•Ǥ–Š‡—Ž‹‡Ž›‡˜‡––Šƒ–ƒ’”‘„Ž‡”‡Žƒ–‹‰–‘‹–‹•ˆ‘—†ǡ’Ž‡ƒ•‡‹ˆ‘”–Š‡‡–”ƒŽ‡…”‡–ƒ”‹ƒ–‘”ƒ––Š‡ƒ††”‡••
‰‹˜‡„‡Ž‘™Ǥ




© ȀʹͲͳͷ
̹ʹͲͳͷ
ŽŽ ”‹‰Š–• ”‡•‡”˜‡†Ǥ Ž‡•• ‘–Š‡”™‹•‡ •’‡…‹ˆ‹‡†ǡ ‘ ’ƒ”– ‘ˆ –Š‹• ’—„Ž‹…ƒ–‹‘ ƒ› „‡ ”‡’”‘†—…‡† ‘” —–‹Ž‹œ‡† ‹ ƒ› ˆ‘” ‘” „› ƒ› ‡ƒ•ǡ
‡Ž‡…–”‘‹… ‘” ‡…Šƒ‹…ƒŽǡ ‹…Ž—†‹‰ ’Š‘–‘…‘’›‹‰ ƒ† ‹…”‘ˆ‹Žǡ ™‹–Š‘—– ’‡”‹••‹‘ ‹ ™”‹–‹‰ ˆ”‘ ǡ  ‘”  ƒ– –Š‡ ”‡•’‡…–‹˜‡
ƒ††”‡••„‡Ž‘™Ǥ
…‘’›”‹‰Š–‘ˆˆ‹…‡
ŠǤ†‡Žƒ†‘‡–ͺȉͶͲͳ
ǦͳʹͳͶ‡”‹‡”ǡ
‡‡˜ƒǡ™‹–œ‡”Žƒ†
‡‡˜ƒʹͲ ͳͲͲͳ͸Ǧͷͻͻ͹ǡ
‡ŽǤΪͶͳʹʹ͹ͶͻͲͳͳͳ
ƒšΪͶͳʹʹ͹ͶͻͲͻͶ͹
…‘’›”‹‰Š–̷‹•‘Ǥ‘”‰
™™™Ǥ‹•‘Ǥ‘”‰
—„Ž‹•Š‡†‹™‹–œ‡”Žƒ†
‡–”ƒŽˆˆ‹…‡
͵ǡ”—‡†‡ƒ”‡„±
Ǧͳʹͳͳ
™‹–œ‡”Žƒ†
Ǧƒ‹Ž‹ƒ‹Ž̷‹‡…Ǥ…Š
‡„™™™Ǥ‹‡…Ǥ…Š
•–‹–—–‡‘ˆŽ‡…–”‹…ƒŽƒ†Ž‡…–”‘‹…•‰‹‡‡”•ǡ…Ǥ͵
ƒ”˜‡—‡ǡ‡™‘”
Ǧƒ‹Ž•–†•Ǥ‹’”̷‹‡‡‡Ǥ‘”‰
‡„™™™Ǥ‹‡‡‡Ǥ‘”‰

̹ȀʹͲͳͷȂŽŽ”‹‰Š–•”‡•‡”˜‡†
ii ̹ʹͲͳͷȂŽŽ”‹‰Š–•”‡•‡”˜‡†

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Contents Page

Foreword ..........................................................................................................................................................................................................................................v
Introduction................................................................................................................................................................................................................................ vi
1 Scope ................................................................................................................................................................................................................................. 1
2 Conformance............................................................................................................................................................................................................. 1
2.1 Intended Usage ....................................................................................................................................................................................... 1
2.2 Full Conformance .................................................................................................................................................................................. 1
2.3 Tailored Conformance ...................................................................................................................................................................... 1
3 Normative References ..................................................................................................................................................................................... 1
Ͷ ‡”•ƒ†‡ϐ‹‹–‹‘• .................................................................................................................................................................................... 2
5 Test Design Techniques ................................................................................................................................................................................. 4
5.1 Overview ...................................................................................................................................................................................................... 4
ͷǤʹ ’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡• ................................................................................................................ 7
5.2.1 Equivalence Partitioning ........................................................................................................................................... 7
ͷǤʹǤʹ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†...................................................................................................................................... 8
ͷǤʹǤ͵ ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹• .......................................................................................................................................... 9
ͷǤʹǤͶ ›–ƒš‡•–‹‰ ................................................................................................................................................................. 11
5.2.5 Combinatorial Test Design Techniques ..................................................................................................... 12
5.2.6 Decision Table Testing.............................................................................................................................................. 15
5.2.7 Cause-Effect Graphing .............................................................................................................................................. 15
5.2.8 State Transition Testing .......................................................................................................................................... 16
5.2.9 Scenario Testing ............................................................................................................................................................ 17
5.2.10 Random Testing ............................................................................................................................................................. 18
5.3 Structure-Based Test Design Techniques ...................................................................................................................... 18
5.3.1 Statement Testing ........................................................................................................................................................ 18
5.3.2 Branch Testing ................................................................................................................................................................ 19
5.3.3 Decision Testing............................................................................................................................................................. 20
5.3.4 Branch Condition Testing ...................................................................................................................................... 20
5.3.5 Branch Condition Combination Testing .................................................................................................... 21
ͷǤ͵Ǥ͸ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡ȋȌ‡•–‹‰ ............................................................ 21
5.3.7 Data Flow Testing ......................................................................................................................................................... 22
5.4 Experience-Based Test Design Techniques ................................................................................................................. 25
5.4.1 Error Guessing ................................................................................................................................................................ 25
6 Test Coverage Measurement .................................................................................................................................................................25
6.1 Overview ................................................................................................................................................................................................... 25
͸Ǥʹ ‡•–‡ƒ•—”‡‡–ˆ‘”’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡• .................................................... 26
6.2.1 Equivalence Partition Coverage ....................................................................................................................... 26
͸ǤʹǤʹ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†‘˜‡”ƒ‰‡ .......................................................................................................... 26
͸ǤʹǤ͵ ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹•‘˜‡”ƒ‰‡............................................................................................................... 26
͸ǤʹǤͶ ›–ƒš‡•–‹‰‘˜‡”ƒ‰‡ ......................................................................................................................................... 26
6.2.5 Combinatorial Test Design Technique Coverage ............................................................................... 27
6.2.6 Decision Table Testing Coverage ..................................................................................................................... 27
6.2.7 Cause-Effect Graphing Coverage ..................................................................................................................... 28
6.2.8 State Transition Testing Coverage ................................................................................................................. 28
6.2.9 Scenario Testing Coverage.................................................................................................................................... 28
6.2.10 Random Testing Coverage .................................................................................................................................... 28
͸Ǥ͵ ‡•–‡ƒ•—”‡‡–ˆ‘”–”—…–—”‡Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡• ............................................................. 29
6.3.1 Statement Testing Coverage ................................................................................................................................ 29
6.3.2 Branch Testing Coverage........................................................................................................................................ 29
6.3.3 Decision Testing Coverage .................................................................................................................................... 29
6.3.4 Branch Condition Testing Coverage ............................................................................................................. 29
6.3.5 Branch Condition Combination Testing Coverage ........................................................................... 29
͸Ǥ͵Ǥ͸ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘ȋȌ‡•–‹‰‘˜‡”ƒ‰‡ ............................................................ 30
© ISO/IEC 2015 – All rights reserved iii
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

6.3.7 Data Flow Testing Coverage ................................................................................................................................ 30

͸ǤͶ ‡•–‡ƒ•—”‡‡–ˆ‘”š’‡”‹‡…‡Ǧƒ•‡†‡•–‹‰‡•‹‰‡…Š‹“—‡•................................................. 31
6.4.1 Error Guessing Coverage ....................................................................................................................................... 31
Annex A (informative) Testing Quality Characteristics ..................................................................................................................32
Annex B (informative)
—‹†‡Ž‹‡•ƒ†šƒ’Ž‡•ˆ‘”–Š‡’’Ž‹…ƒ–‹‘‘ˆ’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†
Test Design Techniques .............................................................................................................................................................................. 43
Annex C (informative) Guidelines and Examples for the Application of Structure-Based
Test Design Techniques .......................................................................................................................................................................... 103
Annex D (informative) Guidelines and Examples for the Application of Experience-Based
Test Design Techniques .......................................................................................................................................................................... 126
Annex E (informative) Guidelines and Examples for the Application of Interchangeable
Test Design Techniques .......................................................................................................................................................................... 129
Annex F (informative) Test Design Technique Coverage Effectiveness...................................................................... 133
Annex G (informative) ISO/IEC/IEEE 29119-4 and BS 7925-2 Test Design Technique Alignment135
Bibliography ......................................................................................................................................................................................................................... 137

iv © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
‘‹••‹‘Ȍ ˆ‘” –Š‡ •’‡…‹ƒŽ‹œ‡† •›•–‡ ˆ‘” ™‘”Ž†™‹†‡ •–ƒ†ƒ”†‹œƒ–‹‘Ǥ ƒ–‹‘ƒŽ „‘†‹‡• –Šƒ– ƒ”‡
members of ISO or IEC participate in the development of International Standards through technical
…‘‹––‡‡• ‡•–ƒ„Ž‹•Š‡† „› –Š‡ ”‡•’‡…–‹˜‡ ‘”‰ƒ‹œƒ–‹‘ –‘ †‡ƒŽ ™‹–Š ’ƒ”–‹…—Žƒ” ϐ‹‡Ž†• ‘ˆ –‡…Š‹…ƒŽ
ƒ…–‹˜‹–›Ǥƒ†–‡…Š‹…ƒŽ…‘‹––‡‡•…‘ŽŽƒ„‘”ƒ–‡‹ϐ‹‡Ž†•‘ˆ—–—ƒŽ‹–‡”‡•–Ǥ–Š‡”‹–‡”ƒ–‹‘ƒŽ
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the
™‘”Ǥ–Š‡ϐ‹‡Ž†‘ˆ‹ˆ‘”ƒ–‹‘–‡…Š‘Ž‘‰›ǡƒ†Šƒ˜‡‡•–ƒ„Ž‹•Š‡†ƒŒ‘‹––‡…Š‹…ƒŽ…‘‹––‡‡ǡ
ISO/IEC JTC 1.
The procedures used to develop this document and those intended for its further maintenance are
described in the ISO/IEC Directives, Part 1. In particular the different approval criteria needed for
–Š‡†‹ˆˆ‡”‡––›’‡•‘ˆ†‘…—‡–•Š‘—Ž†„‡‘–‡†ǤŠ‹•†‘…—‡–™ƒ•†”ƒˆ–‡†‹ƒ……‘”†ƒ…‡™‹–Š–Š‡
editorial rules of the ISO/IEC Directives, Part 2 (see www.iso.org/directives).
––‡–‹‘ ‹• †”ƒ™ –‘ –Š‡ ’‘••‹„‹Ž‹–› –Šƒ– •‘‡ ‘ˆ –Š‡ ‡Ž‡‡–• ‘ˆ –Š‹• †‘…—‡– ƒ› „‡ –Š‡ •—„Œ‡…–
‘ˆ ’ƒ–‡– ”‹‰Š–•Ǥ  ƒ†  •ŠƒŽŽ ‘– „‡ Š‡Ž† ”‡•’‘•‹„Ž‡ ˆ‘” ‹†‡–‹ˆ›‹‰ ƒ› ‘” ƒŽŽ •—…Š ’ƒ–‡–
”‹‰Š–•Ǥ‡–ƒ‹Ž•‘ˆƒ›’ƒ–‡–”‹‰Š–•‹†‡–‹ϐ‹‡††—”‹‰–Š‡†‡˜‡Ž‘’‡–‘ˆ–Š‡†‘…—‡–™‹ŽŽ„‡‹–Š‡
Introduction and/or on the ISO list of patent declarations received (see www.iso.org/patents).
›–”ƒ†‡ƒ‡—•‡†‹–Š‹•†‘…—‡–‹•‹ˆ‘”ƒ–‹‘‰‹˜‡ˆ‘”–Š‡…‘˜‡‹‡…‡‘ˆ—•‡”•ƒ††‘‡•‘–
constitute an endorsement.
‘” ƒ ‡š’Žƒƒ–‹‘ ‘ –Š‡ ‡ƒ‹‰ ‘ˆ  •’‡…‹ϐ‹… –‡”• ƒ† ‡š’”‡••‹‘• ”‡Žƒ–‡† –‘ …‘ˆ‘”‹–›
assessment, as well as information about ISO’s adherence to the WTO principles in the Technical
Barriers to Trade (TBT) see the following URL: ‘”‡™‘”†Ǧ—’’Ž‡‡–ƒ”›‹ˆ‘”ƒ–‹‘
The committee responsible for this document is ISO/IEC JTC 1, Information technology, SC 7, Software
and Systems Engineering.
ISO/IEC/IEEE 29119 consists of the following standards, under the general title Software and Systems
Engineering — Software Testing:
— ƒ”–ͷǣ‘…‡’–•ƒ††‡ϔ‹‹–‹‘•
— Part 2: Test processes
— Part 3: Test documentation
— Part 4: Test techniques
The following parts are under preparation:
— Part 5: Keyword-driven testing

© ISO/IEC 2015 – All rights reserved v

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Introduction
Š‡’—”’‘•‡‘ˆ–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ‹•–‘’”‘˜‹†‡ƒ–‡”ƒ–‹‘ƒŽ–ƒ†ƒ”†–Šƒ–†‡ϐ‹‡•
software test design techniques (also known as test case design techniques or test methods) that can be
—•‡†™‹–Š‹–Š‡–‡•–†‡•‹‰ƒ†‹’Ž‡‡–ƒ–‹‘’”‘…‡••–Šƒ–‹•†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦʹǤŠ‹•
part of ISO/IEC/IEEE 29119 does not prescribe a process for test design and implementation; instead, it
describes a set of techniques that can be used within ISO/IEC/IEEE 29119-2. The intent is to describe a
•‡”‹‡•‘ˆ–‡…Š‹“—‡•–Šƒ–Šƒ˜‡™‹†‡ƒ……‡’–ƒ…‡‹–Š‡•‘ˆ–™ƒ”‡–‡•–‹‰‹†—•–”›Ǥ
The test design techniques presented in this part of ISO/IEC/IEEE 29119 can be used to derive test
cases that, when executed, generate evidence that test item requirements have been met and/or that
defects are present in a test item (i.e. that requirements have not been met). Risk-based testing could be
—•‡†–‘†‡–‡”‹‡–Š‡•‡–‘ˆ–‡…Š‹“—‡•–Šƒ–ƒ”‡ƒ’’Ž‹…ƒ„Ž‡‹•’‡…‹ϐ‹…•‹–—ƒ–‹‘•ȋ”‹•Ǧ„ƒ•‡†–‡•–‹‰‹•
covered in ISO/IEC/IEEE 29119-1 and ISO/IEC/IEEE 29119-2).
NOTE A “test item” is a work product that is being tested (see ISO/IEC/IEEE 29119-1).

ͳ Dz‡•–‹–‡•dz‹…Ž—†‡•›•–‡•ǡ•‘ˆ–™ƒ”‡‹–‡•ǡ‘„Œ‡…–•ǡ…Žƒ••‡•ǡ”‡“—‹”‡‡–•†‘…—‡–•ǡ†‡•‹‰
•’‡…‹ϐ‹…ƒ–‹‘•ǡƒ†—•‡”‰—‹†‡•Ǥ

ƒ…Š –‡…Š‹“—‡ ˆ‘ŽŽ‘™• –Š‡ –‡•– †‡•‹‰ ƒ† ‹’Ž‡‡–ƒ–‹‘ ’”‘…‡•• –Šƒ– ‹• †‡ϐ‹‡† ‹ ȀȀ
IEEE 29119-2 and shown in Figure 1.
Of the activities in this process, ISO/IEC/IEEE 29119-4 provides guidance on how to implement the
following activities in detail for each technique that is described:
— Derive Test Conditions (TD2);
— Derive Test Coverage Items (TD3);
— Derive Test Cases (TD4).
–‡•–…‘†‹–‹‘‹•ƒ–‡•–ƒ„Ž‡ƒ•’‡…–‘ˆƒ–‡•–‹–‡ǡ•—…Šƒ•ƒˆ—…–‹‘ǡ–”ƒ•ƒ…–‹‘ǡˆ‡ƒ–—”‡ǡ“—ƒŽ‹–›ƒ––”‹„—–‡ǡ
‘”•–”—…–—”ƒŽ‡Ž‡‡–‹†‡–‹ϐ‹‡†ƒ•ƒ„ƒ•‹•ˆ‘”–‡•–‹‰ǤŠ‹•†‡–‡”‹ƒ–‹‘…ƒ„‡ƒ…Š‹‡˜‡†„›ƒ‰”‡‡‹‰
™‹–Š•–ƒ‡Š‘Ž†‡”•™Š‹…Šƒ––”‹„—–‡•ƒ”‡–‘„‡–‡•–‡†‘”„›ƒ’’Ž›‹‰‘‡‘”‘”‡–‡•–†‡•‹‰–‡…Š‹“—‡•Ǥ
ʹ ˆƒ–‡•–…‘’Ž‡–‹‘…”‹–‡”‹‘ˆ‘”•–ƒ–‡–”ƒ•‹–‹‘–‡•–‹‰™ƒ•‹†‡–‹ϐ‹‡†–Šƒ–”‡“—‹”‡†…‘˜‡”ƒ‰‡‘ˆ
all states then the test conditions could be the states the test item can be in. Other examples of test conditions are
equivalence classes and boundaries between them.

Test coverage items are attributes of each test condition that can be covered during testing. A single
–‡•–…‘†‹–‹‘ƒ›„‡–Š‡„ƒ•‹•ˆ‘”‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•Ǥ
͵ ˆƒ•’‡…‹ϐ‹…„‘—†ƒ”›‹•‹†‡–‹ϐ‹‡†ƒ•ƒ–‡•–…‘†‹–‹‘ǡ–Š‡–Š‡…‘””‡•’‘†‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•
…‘—Ž†„‡–Š‡„‘—†ƒ”›‹–•‡Žˆƒ†‹‡†‹ƒ–‡Ž›‡‹–Š‡”•‹†‡‘ˆ–Š‡„‘—†ƒ”›Ǥ

A test case is a set of preconditions, inputs (including actions, where applicable), and expected results,
†‡˜‡Ž‘’‡†–‘†‡–‡”‹‡™Š‡–Š‡”‘”‘––Š‡…‘˜‡”‡†’ƒ”–‘ˆ–Š‡–‡•–‹–‡Šƒ•„‡‡‹’Ž‡‡–‡†…‘””‡…–Ž›Ǥ
’‡…‹ϐ‹… ȋ‘”ƒ–‹˜‡Ȍ ‰—‹†ƒ…‡ ‘ Š‘™ –‘ ‹’Ž‡‡– –Š‡ ‘–Š‡” ƒ…–‹˜‹–‹‡• ‹ –Š‡ –‡•– †‡•‹‰ Ƭ
‹’Ž‡‡–ƒ–‹‘’”‘…‡••‘ˆȀȀʹͻͳͳͻǦʹǡ‹…Ž—†‹‰ƒ…–‹˜‹–‹‡•ͳȋ†‡–‹ˆ› ‡ƒ–—”‡‡–•Ȍǡͷ
(Assemble Test Sets), and TD6 (Derive Test Procedures), is not included in Clauses 5 or 6 of this part of
ISO/IEC/IEEE 29119 because the process is the same for all techniques.

vi © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Figure 1 — ISO/IEC/IEEE 29119-2 Test Design and Implementation Process

ȀȀͳͻ͹ͷͻȋȌ†‡ϐ‹‡•–™‘–›’‡•‘ˆ”‡“—‹”‡‡–•ǣˆ—…–‹‘ƒŽ”‡“—‹”‡‡–•ƒ†“—ƒŽ‹–›
”‡“—‹”‡‡–•ǤȀʹͷͲͳͲ†‡ϐ‹‡•‡‹‰Š–“—ƒŽ‹–›…Šƒ”ƒ…–‡”‹•–‹…•ȋ‹…Ž—†‹‰ˆ—…–‹‘ƒŽ‹–›Ȍ–Šƒ–…ƒ„‡
—•‡†–‘‹†‡–‹ˆ›–›’‡•‘ˆ–‡•–‹‰–Šƒ–ƒ›„‡ƒ’’Ž‹…ƒ„Ž‡ˆ‘”–‡•–‹‰ƒ•’‡…‹ϐ‹…–‡•–‹–‡Ǥ‡š’”‘˜‹†‡•
‡šƒ’Ž‡ ƒ’’‹‰• ‘ˆ –‡•– †‡•‹‰ –‡…Š‹“—‡• –Šƒ– ƒ’’Ž› –‘ –‡•–‹‰ “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹…• †‡ϐ‹‡† ‹
ISO/IEC 25010.
š’‡”‹‡…‡Ǧ„ƒ•‡†–‡•–‹‰’”ƒ…–‹…‡•Ž‹‡‡š’Ž‘”ƒ–‘”›–‡•–‹‰ƒ†‘–Š‡”–‡•–’”ƒ…–‹…‡••—…Šƒ•‘†‡ŽǦ„ƒ•‡†
–‡•–‹‰ƒ”‡‘–†‡ϐ‹‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ„‡…ƒ—•‡–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ‘Ž›
†‡•…”‹„‡•–‡…Š‹“—‡•ˆ‘”†‡•‹‰‹‰–‡•–…ƒ•‡•Ǥ‡•–’”ƒ…–‹…‡••—…Šƒ•‡š’Ž‘”ƒ–‘”›–‡•–‹‰ƒ”‡†‡•…”‹„‡†
in ISO/IEC/IEEE 29119-1.
‡’Žƒ–‡•ƒ†‡šƒ’Ž‡•‘ˆ–‡•–†‘…—‡–ƒ–‹‘–Šƒ–ƒ”‡’”‘†—…‡††—”‹‰–Š‡–‡•–‹‰’”‘…‡••ƒ”‡†‡ϐ‹‡†
in ISO/IEC/IEEE 29119-3 Test Documentation. The test techniques in this part of ISO/IEC/IEEE 29119
†‘‘–†‡•…”‹„‡Š‘™–‡•–…ƒ•‡••Š‘—Ž†„‡†‘…—‡–‡†ȋ‡Ǥ‰Ǥ–Š‡›†‘‘–‹…Ž—†‡‹ˆ‘”ƒ–‹‘‘”‰—‹†ƒ…‡
‘ ƒ••‹‰‹‰ —‹“—‡ ‹†‡–‹ϐ‹‡”•ǡ –‡•– …ƒ•‡ †‡•…”‹’–‹‘•ǡ ’”‹‘”‹–‹‡•ǡ –”ƒ…‡ƒ„‹Ž‹–›ǡ ‘” ’”‡Ǧ…‘†‹–‹‘•ȌǤ
Information on how to document test cases can be found in ISO/IEC/IEEE 29119-3.
Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻƒ‹•–‘’”‘˜‹†‡•–ƒ‡Š‘Ž†‡”•™‹–Š–Š‡ƒ„‹Ž‹–›–‘†‡•‹‰–‡•–…ƒ•‡•ˆ‘”
–Š‡–‡•–‹‰‘ˆ•‘ˆ–™ƒ”‡‹ƒ›‘”‰ƒ‹œƒ–‹‘Ǥ

© ISO/IEC 2015 – All rights reserved vii

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 INTERNATIONAL STANDARD ISO/IEC/IEEE 29119-4:2015(E)

Software and systems engineering — Software testing —

Part 4:
Test techniques

1 Scope
Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ†‡ϐ‹‡•–‡•–†‡•‹‰–‡…Š‹“—‡•–Šƒ–…ƒ„‡—•‡††—”‹‰–Š‡–‡•–†‡•‹‰
ƒ†‹’Ž‡‡–ƒ–‹‘’”‘…‡••–Šƒ–‹•†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦʹǤ
This part of ISO/IEC/IEEE 29119 is intended for, but not limited to, testers, test managers, and
†‡˜‡Ž‘’‡”•ǡ’ƒ”–‹…—Žƒ”Ž›–Š‘•‡”‡•’‘•‹„Ž‡ˆ‘”ƒƒ‰‹‰ƒ†‹’Ž‡‡–‹‰•‘ˆ–™ƒ”‡–‡•–‹‰Ǥ

2 Conformance

2.1 Intended Usage

The normative requirements in this part of ISO/IEC/IEEE 29119 are contained in Clauses 5 and 6.
– ‹• ”‡…‘‰‹•‡† –Šƒ– ’ƒ”–‹…—Žƒ” ’”‘Œ‡…–• ‘” ‘”‰ƒ‹œƒ–‹‘• ƒ› ‘– ‡‡† –‘ —•‡ ƒŽŽ ‘ˆ –Š‡ –‡…Š‹“—‡•
†‡ϐ‹‡† „› –Š‹• •–ƒ†ƒ”†Ǥ Š‡”‡ˆ‘”‡ǡ ‹’Ž‡‡–ƒ–‹‘ ‘ˆ –Š‹• •–ƒ†ƒ”† –›’‹…ƒŽŽ› ‹˜‘Ž˜‡• •‡Ž‡…–‹‰
ƒ•‡–‘ˆ–‡…Š‹“—‡••—‹–ƒ„Ž‡ˆ‘”–Š‡’”‘Œ‡…–‘”‘”‰ƒ‹œƒ–‹‘ǤŠ‡”‡ƒ”‡–™‘™ƒ›•–Šƒ–ƒ‘”‰ƒ‹œƒ–‹‘
or individual can claim conformance to the provisions of this standard – full conformance or tailored
conformance. The organization or individual shall assert whether full or tailored conformance to this
standard is claimed.

2.2 Full Conformance

—ŽŽ…‘ˆ‘”ƒ…‡‹•ƒ…Š‹‡˜‡†„›†‡‘•–”ƒ–‹‰–Šƒ–ƒŽŽ‘ˆ–Š‡”‡“—‹”‡‡–•ȋ‹Ǥ‡ǤǮ•ŠƒŽŽǯ•–ƒ–‡‡–•Ȍ‘ˆ–Š‡
…Š‘•‡ȋ‘Ǧ‡’–›Ȍ•‡–‘ˆ–‡…Š‹“—‡•‹Clause 5 and/or the corresponding test coverage measurement
approaches in Clause 6Šƒ˜‡„‡‡•ƒ–‹•ϐ‹‡†Ǥ
 ‘”‰ƒ‹œƒ–‹‘…‘—Ž†…Š‘‘•‡–‘…‘ˆ‘”‘Ž›–‘‘‡–‡…Š‹“—‡ǡ•—…Šƒ•„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•Ǥ
–Š‹••…‡ƒ”‹‘ǡ–Š‡‘”‰ƒ‹œƒ–‹‘™‘—Ž†‘Ž›„‡”‡“—‹”‡†–‘’”‘˜‹†‡‡˜‹†‡…‡–Šƒ––Š‡›Šƒ˜‡‡––Š‡”‡“—‹”‡‡–•
of that one technique in order to claim conformance to this part of ISO/IEC/IEEE 29119.

2.3 Tailored Conformance

ƒ‹Ž‘”‡†…‘ˆ‘”ƒ…‡‹•ƒ…Š‹‡˜‡†„›†‡‘•–”ƒ–‹‰–Šƒ––Š‡…Š‘•‡•—„•‡–‘ˆ”‡“—‹”‡‡–•ˆ”‘–Š‡
…Š‘•‡ ȋ‘Ǧ‡’–›Ȍ •‡– ‘ˆ –‡…Š‹“—‡• ƒ†Ȁ‘” …‘””‡•’‘†‹‰ –‡•– …‘˜‡”ƒ‰‡ ‡ƒ•—”‡‡– ƒ’’”‘ƒ…Š‡•
Šƒ˜‡ „‡‡ •ƒ–‹•ϐ‹‡†Ǥ Š‡”‡ –ƒ‹Ž‘”‹‰ ‘……—”•ǡ Œ—•–‹ϐ‹…ƒ–‹‘ •ŠƒŽŽ „‡ ’”‘˜‹†‡† ȋ‡‹–Š‡” †‹”‡…–Ž› ‘” „›
”‡ˆ‡”‡…‡Ȍ ™Š‡‡˜‡” –Š‡ ‘”ƒ–‹˜‡ ”‡“—‹”‡‡–• ‘ˆ ƒ –‡…Š‹“—‡ †‡ϐ‹‡† ‹ Clause 5 or measure
†‡ϐ‹‡† ‹ Clause 6 ƒ”‡ ‘– ˆ‘ŽŽ‘™‡† …‘’Ž‡–‡Ž›Ǥ ŽŽ –ƒ‹Ž‘”‹‰ †‡…‹•‹‘• •ŠƒŽŽ „‡ ”‡…‘”†‡† ™‹–Š –Š‡‹”
”ƒ–‹‘ƒŽ‡ǡ‹…Ž—†‹‰–Š‡…‘•‹†‡”ƒ–‹‘‘ˆƒ›ƒ’’Ž‹…ƒ„Ž‡”‹••Ǥƒ‹Ž‘”‹‰•ŠƒŽŽ„‡ƒ‰”‡‡†„›–Š‡”‡Ž‡˜ƒ–
stakeholders.

3 Normative References
Š‡ ˆ‘ŽŽ‘™‹‰ †‘…—‡–•ǡ ‹ ™Š‘Ž‡ ‘” ‹ ’ƒ”–ǡ ƒ”‡ ‘”ƒ–‹˜‡Ž› ”‡ˆ‡”‡…‡† ‹ –Š‹• †‘…—‡– ƒ† ƒ”‡
‹†‹•’‡•ƒ„Ž‡ ˆ‘” ‹–• ƒ’’Ž‹…ƒ–‹‘Ǥ ‘” †ƒ–‡† ”‡ˆ‡”‡…‡•ǡ ‘Ž› –Š‡ ‡†‹–‹‘ …‹–‡† ƒ’’Ž‹‡•Ǥ ‘” —†ƒ–‡†
”‡ˆ‡”‡…‡•ǡ–Š‡Žƒ–‡•–‡†‹–‹‘‘ˆ–Š‡”‡ˆ‡”‡…‡††‘…—‡–ȋ‹…Ž—†‹‰ƒ›ƒ‡†‡–•Ȍƒ’’Ž‹‡•Ǥ

© ISO/IEC 2015 – All rights reserved 1

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

ISO/IEC/IEEE 29119-1, Software and systems engineering — Software testing — Part 1: Concepts and
†‡ϔ‹‹–‹‘•
ISO/IEC/IEEE 29119-2, Software and systems engineering — Software testing — Part 2: Test processes
ISO/IEC/IEEE 29119-3, Software and systems engineering — Software testing — Part 3: Test
documentation
NOTE Other International Standards useful for the implementation and interpretation of this part of
ȀȀʹͻͳͳͻƒ”‡Ž‹•–‡†‹–Š‡„‹„Ž‹‘‰”ƒ’Š›Ǥ

Ͷ ‡”•ƒ†‡ϐ‹‹–‹‘•
‘” –Š‡ ’—”’‘•‡• ‘ˆ –Š‹• †‘…—‡–ǡ –Š‡ –‡”• ƒ† †‡ϐ‹‹–‹‘• ‰‹˜‡ ‹ ȀȀ ʹͶ͹͸ͷ ƒ† –Š‡
ˆ‘ŽŽ‘™‹‰ƒ’’Ž›Ǥ
 •‡ ‘ˆ –Š‡ –‡”‹‘Ž‘‰› ‹ –Š‹• ’ƒ”– ‘ˆ ȀȀ ʹͻͳͳͻ ‹• ˆ‘” ‡ƒ•‡ ‘ˆ ”‡ˆ‡”‡…‡ ƒ† ‹• ‘–
ƒ†ƒ–‘”›ˆ‘”…‘ˆ‘”ƒ…‡™‹–Š–Š‡•–ƒ†ƒ”†ǤŠ‡ˆ‘ŽŽ‘™‹‰–‡”•ƒ††‡ϐ‹‹–‹‘•ƒ”‡’”‘˜‹†‡†–‘ƒ••‹•–™‹–Š
–Š‡—†‡”•–ƒ†‹‰ƒ†”‡ƒ†ƒ„‹Ž‹–›‘ˆ–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻǤŽ›–‡”•…”‹–‹…ƒŽ–‘–Š‡—†‡”•–ƒ†‹‰
of this part of ISO/IEC/IEEE 29119 are included. This clause is not intended to provide a complete list of testing
–‡”•ǤŠ‡•›•–‡•ƒ†•‘ˆ–™ƒ”‡‡‰‹‡‡”‹‰˜‘…ƒ„—Žƒ”›ȀȀʹͶ͹͸ͷ…ƒ„‡”‡ˆ‡”‡…‡†ˆ‘”–‡”•‘–
†‡ϐ‹‡†‹–Š‹•…Žƒ—•‡Ǥ

4.1
Backus-Naur Form
ˆ‘”ƒŽ‡–ƒǦŽƒ‰—ƒ‰‡—•‡†ˆ‘”†‡ϐ‹‹‰–Š‡•›–ƒš‘ˆƒŽƒ‰—ƒ‰‡‹ƒ–‡š–—ƒŽˆ‘”ƒ–
4.2
base choice
see base value
4.3
base value
‹’—– ’ƒ”ƒ‡–‡” ˜ƒŽ—‡ —•‡† ‹ Ǯ„ƒ•‡ …Š‘‹…‡ –‡•–‹‰ǯ –Šƒ– ‹• ‘”ƒŽŽ› •‡Ž‡…–‡† „ƒ•‡† ‘ „‡‹‰ ƒ
”‡’”‡•‡–ƒ–‹˜‡‘”–›’‹…ƒŽ˜ƒŽ—‡ˆ‘”–Š‡’ƒ”ƒ‡–‡”ǤŽ•‘…ƒŽŽ‡†„ƒ•‡…Š‘‹…‡
4.4
c-use
see computation data use
4.5
computation data use
—•‡‘ˆ–Š‡˜ƒŽ—‡‘ˆƒ˜ƒ”‹ƒ„Ž‡‹ƒ›–›’‡‘ˆ•–ƒ–‡‡–
4.6
condition
Boolean expression containing no Boolean operators
 Dzδdz‹•ƒ…‘†‹–‹‘„—–Dzƒ†dz‹•‘–Ǥ

4.7
…‘–”‘ŽϐŽ‘™
sequence in which operations are performed during the execution of a test item
4.8
…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š
sequence of executable statements within a test item
4.9
†ƒ–ƒ†‡ϐ‹‹–‹‘
•–ƒ–‡‡–™Š‡”‡ƒ˜ƒ”‹ƒ„Ž‡‹•ƒ••‹‰‡†ƒ˜ƒŽ—‡ǤŽ•‘…ƒŽŽ‡†˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘

2 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

4.10
†ƒ–ƒ†‡ϐ‹‹–‹‘…Ǧ—•‡’ƒ‹”
†ƒ–ƒ†‡ϐ‹‹–‹‘ƒ†•—„•‡“—‡–…‘’—–ƒ–‹‘†ƒ–ƒ—•‡ǡ™Š‡”‡–Š‡†ƒ–ƒ—•‡—•‡•–Š‡˜ƒŽ—‡†‡ϐ‹‡†‹–Š‡
†ƒ–ƒ†‡ϐ‹‹–‹‘
4.11
†ƒ–ƒ†‡ϐ‹‹–‹‘’Ǧ—•‡’ƒ‹”
†ƒ–ƒ †‡ϐ‹‹–‹‘ ƒ† •—„•‡“—‡– ’”‡†‹…ƒ–‡ †ƒ–ƒ —•‡ǡ ™Š‡”‡ –Š‡ †ƒ–ƒ —•‡ —•‡• –Š‡ ˜ƒŽ—‡ †‡ϐ‹‡† ‹ –Š‡
†ƒ–ƒ†‡ϐ‹‹–‹‘
4.12
†ƒ–ƒ†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”
†ƒ–ƒ†‡ϐ‹‹–‹‘ƒ†•—„•‡“—‡–†ƒ–ƒ—•‡ǡ™Š‡”‡–Š‡†ƒ–ƒ—•‡—•‡•–Š‡˜ƒŽ—‡†‡ϐ‹‡†‹–Š‡†ƒ–ƒ†‡ϐ‹‹–‹‘
4.13
data use
executable statement where the value of a variable is accessed
4.14
decision outcome
”‡•—Ž–‘ˆƒ†‡…‹•‹‘ȋ™Š‹…Š–Š‡”‡ˆ‘”‡†‡–‡”‹‡•–Š‡…‘–”‘ŽϐŽ‘™ƒŽ–‡”ƒ–‹˜‡–ƒ‡Ȍ
4.15
decision rule
combination of conditions (also known as causes) and actions (also known as effects) that produce a
•’‡…‹ϐ‹…‘—–…‘‡‹†‡…‹•‹‘–ƒ„Ž‡–‡•–‹‰ƒ†…ƒ—•‡Ǧ‡ˆˆ‡…–‰”ƒ’Š‹‰
4.16
†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”
†ƒ–ƒ†‡ϐ‹‹–‹‘ƒ†•—„•‡“—‡–’”‡†‹…ƒ–‡‘”…‘’—–ƒ–‹‘ƒŽ†ƒ–ƒ—•‡ǡ™Š‡”‡–Š‡†ƒ–ƒ—•‡—•‡•–Š‡˜ƒŽ—‡
†‡ϐ‹‡†‹–Š‡†ƒ–ƒ†‡ϐ‹‹–‹‘
4.17
†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ–Š
…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šˆ”‘ƒ˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘ƒ’”‡†‹…ƒ–‡Ǧ—•‡ȋ’Ǧ—•‡Ȍ‘”…‘’—–ƒ–‹‘ƒŽǦ—•‡ȋ…Ǧ—•‡Ȍ
of that variable
4.18
entry point
point in a test item at which execution of the test item can begin
‘–‡ͳ–‘‡–”›ǣ‡–”›’‘‹–‹•ƒ‡š‡…—–ƒ„Ž‡•–ƒ–‡‡–™‹–Š‹ƒ–‡•–‹–‡–Šƒ–ƒ›„‡•‡Ž‡…–‡†„›ƒ‡š–‡”ƒŽ
’”‘…‡••ƒ•–Š‡•–ƒ”–‹‰’‘‹–ˆ‘”‘‡‘”‘”‡’ƒ–Š•–Š”‘—‰Š–Š‡–‡•–‹–‡Ǥ–‹•‘•–…‘‘Ž›–Š‡ϐ‹”•–‡š‡…—–ƒ„Ž‡
statement within the test item.

4.19
executable statement
•–ƒ–‡‡–™Š‹…Šǡ™Š‡…‘’‹Ž‡†ǡ‹•–”ƒ•Žƒ–‡†‹–‘‘„Œ‡…–…‘†‡ǡ™Š‹…Š™‹ŽŽ„‡‡š‡…—–‡†’”‘…‡†—”ƒŽŽ›
™Š‡–Š‡–‡•–‹–‡‹•”—‹‰ƒ†ƒ›’‡”ˆ‘”ƒƒ…–‹‘‘’”‘‰”ƒ†ƒ–ƒ
4.20
exit point
last executable statement within a test item
‘–‡ͳ–‘‡–”›ǣ‡š‹–’‘‹–‹•ƒ–‡”‹ƒŽ’‘‹–‘ˆƒ’ƒ–Š–Š”‘—‰Šƒ–‡•–‹–‡ǡ„‡‹‰ƒ‡š‡…—–ƒ„Ž‡•–ƒ–‡‡–
within the test item which either terminates the test item, or returns control to an external process. This is most
…‘‘Ž›–Š‡Žƒ•–‡š‡…—–ƒ„Ž‡•–ƒ–‡‡–™‹–Š‹–Š‡–‡•–‹–‡Ǥ

4.21
p-use
see predicate data use

© ISO/IEC 2015 – All rights reserved 3

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

4.22
P-V pair
combination of a test item parameter with a value assigned to that parameter, used as a test condition
and coverage item in combinatorial test design techniques
4.23
path
sequence of executable statements of a test item
4.24
predicate
Ž‘‰‹…ƒŽ‡š’”‡••‹‘™Š‹…Š‡˜ƒŽ—ƒ–‡•–‘‘” ǡ‘”ƒŽŽ›–‘†‹”‡…––Š‡‡š‡…—–‹‘’ƒ–Š‹…‘†‡
4.25
predicate data use
data use associated with the decision outcome of the predicate portion of a decision statement
4.26
sub-path
path that is part of a larger path
4.27
test model
representation of a test item that is used during the test case design process
4.28
˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘
•‡‡†ƒ–ƒ†‡ϐ‹‹–‹‘

5 Test Design Techniques

5.1 Overview
ȀȀʹͻͳͳͻǦͶ†‡ϐ‹‡•–‡•–†‡•‹‰–‡…Š‹“—‡•ˆ‘”•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†–‡•–‹‰ȋ5.2), structure-
based testing (5.3) and experience-based testing (5.4ȌǤ  •’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† –‡•–‹‰ǡ –Š‡ –‡•– „ƒ•‹•
ȋ‡Ǥ‰Ǥ ”‡“—‹”‡‡–•ǡ •’‡…‹ϐ‹…ƒ–‹‘•ǡ ‘†‡Ž• ‘” —•‡” ‡‡†•Ȍ ‹• —•‡† ƒ• –Š‡ ƒ‹ •‘—”…‡ ‘ˆ ‹ˆ‘”ƒ–‹‘
to design test cases. In structure-based testing, the structure of the test item (e.g. source code or the
•–”—…–—”‡‘ˆƒ‘†‡ŽȌ‹•—•‡†ƒ•–Š‡’”‹ƒ”›•‘—”…‡‘ˆ‹ˆ‘”ƒ–‹‘–‘†‡•‹‰–‡•–…ƒ•‡•Ǥ‡š’‡”‹‡…‡Ǧ
„ƒ•‡†–‡•–‹‰ǡ–Š‡‘™Ž‡†‰‡ƒ†‡š’‡”‹‡…‡‘ˆ–Š‡–‡•–‡”‹•—•‡†ƒ•–Š‡’”‹ƒ”›•‘—”…‡‘ˆ‹ˆ‘”ƒ–‹‘
†—”‹‰–‡•–…ƒ•‡†‡•‹‰Ǥ ‘”•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†–‡•–‹‰ǡ•–”—…–—”‡Ǧ„ƒ•‡†–‡•–‹‰ƒ†‡š’‡”‹‡…‡Ǧ„ƒ•‡†
testing, the test basis is used to generate the expected results. These classes of test design techniques
ƒ”‡…‘’Ž‡‡–ƒ”›ƒ†–Š‡‹”…‘„‹‡†ƒ’’Ž‹…ƒ–‹‘–›’‹…ƒŽŽ›”‡•—Ž–•‹‘”‡‡ˆˆ‡…–‹˜‡–‡•–‹‰Ǥ
Ž–Š‘—‰Š –Š‡ –‡…Š‹“—‡• ’”‡•‡–‡† ‹ ȀȀ ʹͻͳͳͻǦͶ ƒ”‡ …Žƒ••‹ϐ‹‡† ƒ• •–”—…–—”‡Ǧ„ƒ•‡†ǡ
•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†‘”‡š’‡”‹‡…‡Ǧ„ƒ•‡†ǡ‹’”ƒ…–‹…‡•‘‡‘ˆ–Š‡…ƒ„‡—•‡†‹–‡”…Šƒ‰‡ƒ„Ž›ȋ‡Ǥ‰Ǥ„”ƒ…Š
testing could be used to design test cases for testing logical paths through the graphical user interface
‘ˆƒ–‡”‡–Ǧ„ƒ•‡†•›•–‡ȌǤŠ‹•‹•†‡‘•–”ƒ–‡†‹‡šǤƒ††‹–‹‘ǡƒŽ–Š‘—‰Š‡ƒ…Š–‡…Š‹“—‡‹•
†‡ϐ‹‡†‹†‡’‡†‡–Ž›‘ˆƒŽŽ‘–Š‡”•ǡ‹’”ƒ…–‹…‡–Š‡›…ƒ„‡—•‡†‹…‘„‹ƒ–‹‘™‹–Š‘–Š‡”–‡…Š‹“—‡•Ǥ
 Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•†‡”‹˜‡†„›ƒ’’Ž›‹‰‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰…‘—Ž†„‡—•‡†–‘’‘’—Žƒ–‡
input parameters of test cases derived using scenario testing.

ȀȀ ʹͻͳͳͻǦͶ —•‡• –Š‡ –‡”• •’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† –‡•–‹‰ ƒ† •–”—…–—”‡Ǧ„ƒ•‡† –‡•–‹‰Ǣ –Š‡•‡
categories of techniques are also known as “black-box testing” and “white-box testing” (or “clear-box
–‡•–‹‰dzȌ ”‡•’‡…–‹˜‡Ž›Ǥ Š‡ –‡”• Dz„Žƒ…Ǧ„‘šdz ƒ† Dz™Š‹–‡Ǧ„‘šdz ”‡ˆ‡” –‘ –Š‡ ˜‹•‹„‹Ž‹–› ‘ˆ –Š‡ ‹–‡”ƒŽ
structure of the test item. In black-box testing the internal structure of the test item is not visible (hence
the black box), whereas for white-box testing the internal structure of the test item is visible. When a
–‡…Š‹“—‡‹•ƒ’’Ž‹‡†™Š‹Ž‡—–‹Ž‹•‹‰ƒ…‘„‹ƒ–‹‘‘ˆ‘™Ž‡†‰‡ˆ”‘–Š‡–‡•–‹–‡ǯ••’‡…‹ϐ‹…ƒ–‹‘ƒ†
•–”—…–—”‡ǡ–Š‹•‹•‘ˆ–‡…ƒŽŽ‡†Dz‰”‡›Ǧ„‘š–‡•–‹‰dzǤ

4 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

ȀȀ ʹͻͳͳͻǦͶ †‡ϐ‹‡• Š‘™ –Š‡ ‰‡‡”‹… –‡•– †‡•‹‰ ƒ† ‹’Ž‡‡–ƒ–‹‘ ’”‘…‡•• •–‡’• ʹ
(derive test conditions), TD3 (derive test coverage items), and TD4 (derive test cases) from ISO/IEC/
ʹͻͳͳͻǦʹȋ•‡‡–”‘†—…–‹‘Ȍ•ŠƒŽŽ„‡—•‡†„›‡ƒ…Š–‡…Š‹“—‡Ǥ–†‘‡•‘–’”‘˜‹†‡…‘–‡š–Ǧ•’‡…‹ϐ‹…
†‡ϐ‹‹–‹‘•‘ˆ–Š‡–‡…Š‹“—‡•–Šƒ–†‡•…”‹„‡Š‘™‡ƒ…Š–‡…Š‹“—‡•Š‘—Ž†„‡—•‡†‹ƒŽŽ•‹–—ƒ–‹‘•Ǥ•‡”•‘ˆ
ȀȀʹͻͳͳͻǦͶƒ›”‡ˆ‡”–‘‡šǡ‡šǡ‡šƒ†‡šˆ‘”†‡–ƒ‹Ž‡†‡šƒ’Ž‡•–Šƒ–
†‡‘•–”ƒ–‡Š‘™–‘ƒ’’Ž›–Š‡–‡…Š‹“—‡•Ǥ
Š‡–‡…Š‹“—‡•–Šƒ–ƒ”‡†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦͶƒ”‡•Š‘™‹ ‹‰—”‡ʹǤŠ‹••‡–‘ˆ–‡…Š‹“—‡•
‹•‘–‡šŠƒ—•–‹˜‡ǤŠ‡”‡ƒ”‡–‡…Š‹“—‡•–Šƒ–ƒ”‡—•‡†„›–‡•–‹‰’”ƒ…–‹–‹‘‡”•‘””‡•‡ƒ”…Š‡”•–Šƒ–ƒ”‡
not included in ISO/IEC/IEEE 29119-4.

© ISO/IEC 2015 – All rights reserved 5

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Test Design Techniques Presented

in ISO/IEC/IEEE 29119-4

Speciication-Based Structure-Based Experience-Based

Techniques Techniques Techniques
(clause 5.2) (clause 5.3) (clause 5.4)

Equivalence Partitioning Statement Testing Error Guessing

(clause 5.2.1) (clause 5.3.1) (clause 5.4.1)

Classiication Tree Method Branch Testing

(clause 5.2.2) (clause 5.3.2)

Boundary Value Analysis Decision Testing

(clause 5.2.3) (clause 5.3.3)

Syntax Testing Branch Condition Testing

(clause 5.2.4) (clause 5.3.4)

Combinatorial Branch Condition

Test Design Techniques Combination Testing
(clause 5.2.5) (clause 5.3.5)

Modiied Condition Decision

All Combinations Testing
Coverage Testing
(clause 5.2.5.3)
(clause 5.3.6)

Pair-Wise Testing Data Flow Testing

(clause 5.2.5.4) (clause 5.3.7)

Each Choice Testing All-Deinitions Testing

(clause 5.2.5.5) (clause 5.3.7.2)

Base Choice Testing All-C-Uses Testing

(clause 5.2.5.6) (clause 5.3.7.3)

Decision Table Testing All-P-Uses Testing

(clause 5.2.6) (clause 5.3.7.4)

Cause-Effect Graphing All-Uses Testing

(clause 5.2.7) (clause 5.3.7.5)

State Transition Testing All-DU-Paths Testing

(clause 5.2.8) (clause 5.3.7.6)

Scenario Testing
(clause 5.2.9)

Figure 2 — The set of test design techniques presented in ISO/IEC/IEEE 29119-4

Of the six activities in the test design and implementation process (see Figure 1), test design techniques
’”‘˜‹†‡ —‹“—‡ ƒ† •’‡…‹ϐ‹… ‰—‹†ƒ…‡ ‘ –Š‡ †‡”‹˜ƒ–‹‘ ‘ˆ –‡•– …‘†‹–‹‘• ȋʹȌǡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡•
ȋ͵Ȍƒ†–‡•–…ƒ•‡•ȋͶȌǤŠ‡”‡ˆ‘”‡ǡ‡ƒ…Š–‡…Š‹“—‡‹•†‡ϐ‹‡†‹–‡”•‘ˆ–Š‡•‡–Š”‡‡ƒ…–‹˜‹–‹‡•Ǥ
Š‡”‡ ƒ”‡ ˜ƒ”›‹‰ Ž‡˜‡Ž• ‘ˆ ‰”ƒ—Žƒ”‹–› ™‹–Š‹ •–‡’• ʹ ȋ†‡”‹˜‡ –‡•– …‘†‹–‹‘•Ȍǡ ͵ ȋ†‡”‹˜‡ –‡•–
coverage items) and TD4 (derive test cases). Within each technique, the term “model” is used to
6 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

describe the concept of preparing a logical representation of the test item for the purposes of deriving
–‡•– …‘†‹–‹‘• ‹ •–‡’ ʹ ȋ‡Ǥ‰Ǥ ƒ …‘–”‘Ž ϐŽ‘™ ‘†‡Ž ‹• ”‡“—‹”‡† ˆ‘” †‡”‹˜‹‰ –‡•– …‘†‹–‹‘• ˆ‘” ƒŽŽ
•–”—…–—”ƒŽ–‡…Š‹“—‡•ȌǤ‘‡•‹–—ƒ–‹‘•ƒ›”‡“—‹”‡–Š‡‡–‹”‡‘†‡Ž–‘„‡ƒ–‡•–…‘†‹–‹‘ǡ™Š‡”‡ƒ•‹
‘–Š‡”•‹–—ƒ–‹‘•ǡ‘‡’ƒ”–‘ˆ–Š‡‘†‡Žƒ›„‡ƒ–‡•–…‘†‹–‹‘Ǥ
ͳ •–ƒ–‡–”ƒ•‹–‹‘–‡•–‹‰ǡ‹ˆ–Š‡”‡‹•ƒ”‡“—‹”‡‡––‘…‘˜‡”ƒŽŽ•–ƒ–‡•–Š‡–Š‡‡–‹”‡•–ƒ–‡‘†‡Ž
…‘—Ž†„‡–Š‡–‡•–…‘†‹–‹‘ǤŽ–‡”ƒ–‹˜‡Ž›ǡ‹ˆ–Š‡”‡‹•ƒ”‡“—‹”‡‡––‘…‘˜‡”•’‡…‹ϐ‹…–”ƒ•‹–‹‘•„‡–™‡‡•–ƒ–‡•ǡ
then each transition could be a test condition.

ƒ††‹–‹‘ǡ•‹…‡•‘‡–‡…Š‹“—‡••Šƒ”‡—†‡”Ž›‹‰…‘…‡’–•ǡ–Š‡‹”†‡ϐ‹‹–‹‘•…‘–ƒ‹•‹‹Žƒ”–‡š–Ǥ
ʹ ‘–Š‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰ƒ†„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ƒ”‡„ƒ•‡†‘‡“—‹˜ƒŽ‡…‡…Žƒ••‡•Ǥ

–Š‡–‡•–…ƒ•‡†‡•‹‰•–‡’ȋͶȌ‘ˆ‡ƒ…Š–‡…Š‹“—‡ǡ–‡•–…ƒ•‡•–Šƒ–ƒ”‡…”‡ƒ–‡†ƒ›„‡Dz˜ƒŽ‹†dzȋ‹Ǥ‡Ǥ–Š‡›
…‘–ƒ‹‹’—–˜ƒŽ—‡•–Šƒ––Š‡–‡•–‹–‡•Š‘—Ž†ƒ……‡’–ƒ•…‘””‡…–Ȍ‘”Dz‹˜ƒŽ‹†dzȋ‹Ǥ‡Ǥ–Š‡›…‘–ƒ‹ƒ–Ž‡ƒ•–
‘‡‹’—–˜ƒŽ—‡–Šƒ––Š‡–‡•–‹–‡•Š‘—Ž†”‡Œ‡…–ƒ•‹…‘””‡…–ǡ‹†‡ƒŽŽ›™‹–Šƒƒ’’”‘’”‹ƒ–‡‡””‘”‡••ƒ‰‡ȌǤ
•‘‡–‡…Š‹“—‡•ǡ•—…Šƒ•‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰ƒ†„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ǡ‹˜ƒŽ‹†–‡•–…ƒ•‡•
ƒ”‡—•—ƒŽŽ›†‡”‹˜‡†—•‹‰–Š‡Dz‘‡Ǧ–‘Ǧ‘‡dzƒ’’”‘ƒ…Šƒ•‹–ƒ˜‘‹†•ˆƒ—Ž–ƒ•‹‰„›‡•—”‹‰–Šƒ–‡ƒ…Š
–‡•– …ƒ•‡ ‘Ž› ‹…Ž—†‡• ‘‡ ‹˜ƒŽ‹† ‹’—– ˜ƒŽ—‡ǡ ™Š‹Ž‡ ˜ƒŽ‹† –‡•– …ƒ•‡• ƒ”‡ –›’‹…ƒŽŽ› †‡”‹˜‡† —•‹‰ –Š‡
“minimized” approach, as this reduces the number of test cases required to cover valid test coverage
items (see 5.2.1.3 and 5.2.3.3).
NOTE Invalid cases are also known as “negative test cases”.

Ž–Š‘—‰Š–Š‡–‡…Š‹“—‡•†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦͶƒ”‡‡ƒ…Š†‡•…”‹„‡†‹ƒ•‡’ƒ”ƒ–‡…Žƒ—•‡ȋƒ•‹ˆ

–Š‡›™‡”‡—–—ƒŽŽ›‡š…Ž—•‹˜‡Ȍǡ‹’”ƒ…–‹…‡–Š‡›…‘—Ž†„‡ƒ’’Ž‹‡†‹ƒ„Ž‡†‡†™ƒ›Ǥ
͵ ‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•…‘—Ž†„‡—•‡†–‘•‡Ž‡…––‡•–‹’—–˜ƒŽ—‡•ǡƒˆ–‡”™Š‹…Š’ƒ‹”Ǧ™‹•‡–‡•–‹‰
could be used to design test cases from the test input values. Equivalence partitioning could be used to select
–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ†…Žƒ••‡•ˆ‘”–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†ƒ†–Š‡‡ƒ…Š…Š‘‹…‡–‡•–‹‰…‘—Ž†„‡—•‡†–‘
construct test cases from the classes.

Š‡ –‡…Š‹“—‡• ’”‡•‡–‡† ‹ ȀȀ ʹͻͳͳͻǦͶ …‘—Ž† ƒŽ•‘ „‡ —•‡† ‹ …‘Œ—…–‹‘ ™‹–Š –Š‡ –‡•–
–›’‡•–Šƒ–ƒ”‡’”‡•‡–‡†‹‡šǤ ‘”‡šƒ’Ž‡ǡ‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰…‘—Ž†„‡—•‡†–‘‹†‡–‹ˆ›
user groups (test conditions) and representative users (test coverage items) from those groups in test
…ƒ•‡•–Šƒ–ƒ”‡–‘„‡–‡•–‡††—”‹‰—•ƒ„‹Ž‹–›–‡•–‹‰Ǥ
Š‡ ‘”ƒ–‹˜‡ †‡ϐ‹‹–‹‘• ‘ˆ –Š‡ –‡…Š‹“—‡• ƒ”‡ ’”‘˜‹†‡† ‹ Clause 5. The corresponding normative
coverage measures for each technique are presented in Clause 6Ǥ Š‹• ‹• •—’’‘”–‡† „› ‹ˆ‘”ƒ–‹˜‡
examples of each technique in Annexes B, C, D and E. Although the examples of each technique
demonstrate manual application of the technique, in practice, automation can be used to support some
–›’‡•‘ˆ–‡•–†‡•‹‰ƒ†‡š‡…—–‹‘ȋ‡Ǥ‰Ǥ•–ƒ–‡‡–…‘˜‡”ƒ‰‡ƒƒŽ›œ‡”•…ƒ„‡—•‡†–‘•—’’‘”–•–”—…–—”‡Ǧ
„ƒ•‡†–‡•–‹‰ȌǤ‡š’”‘˜‹†‡•‡šƒ’Ž‡•‘ˆŠ‘™–Š‡–‡•–†‡•‹‰–‡…Š‹“—‡•†‡ϐ‹‡†‹–Š‹••–ƒ†ƒ”†
…ƒ„‡ƒ’’Ž‹‡†–‘–‡•–‹‰–Š‡“—ƒŽ‹–›…Šƒ”ƒ…–‡”‹•–‹…•–Šƒ–ƒ”‡†‡ϐ‹‡†‹ȀʹͷͲͳͲǤ

ͷǤʹ ’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡•

5.2.1 Equivalence Partitioning

5.2.1.1 Derive Test Conditions (TD2)

“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺǢ›‡”•ͳͻ͹ͻȌ—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–’ƒ”–‹–‹‘•–Š‡
inputs and outputs of the test item into equivalence partitions (also called “partitions” or “equivalence
…Žƒ••‡•dzȌǡ ™Š‡”‡ ‡ƒ…Š ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘ •ŠƒŽŽ „‡ †‡ϐ‹‡† ƒ• ƒ –‡•– …‘†‹–‹‘Ǥ Š‡•‡ ‡“—‹˜ƒŽ‡…‡
partitions shall be derived from the test basis, where each partition is chosen such that all values
™‹–Š‹–Š‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘…ƒ”‡ƒ•‘ƒ„Ž›„‡‡š’‡…–‡†–‘„‡–”‡ƒ–‡†•‹‹Žƒ”Ž›ȋ‹Ǥ‡Ǥ–Š‡›ƒ›„‡

© ISO/IEC 2015 – All rights reserved 7

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

…‘•‹†‡”‡† Dz‡“—‹˜ƒŽ‡–dzȌ „› –Š‡ –‡•– ‹–‡Ǥ “—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘• ƒ› „‡ †‡”‹˜‡† ˆ‘” „‘–Š ˜ƒŽ‹† ƒ†
invalid inputs and outputs.
 ‘” ƒ –‡•– ‹–‡ ‡š’‡…–‹‰ Ž‘™‡”…ƒ•‡ ƒŽ’Šƒ„‡–‹…ƒŽ …Šƒ”ƒ…–‡”• ƒ• ȋ˜ƒŽ‹†Ȍ ‹’—–•ǡ ‹˜ƒŽ‹† ‹’—–
equivalence partitions that could be derived include equivalence partitions containing integers, reals, uppercase
ƒŽ’Šƒ„‡–‹…ƒŽ…Šƒ”ƒ…–‡”•ǡ•›„‘Ž•ƒ†…‘–”‘Ž…Šƒ”ƒ…–‡”•ǡ†‡’‡†‹‰‘–Š‡Ž‡˜‡Ž‘ˆ”‹‰‘—””‡“—‹”‡††—”‹‰–‡•–‹‰Ǥ

NOTE 1 For output equivalence partitions, corresponding input partitions are derived based on the processing
†‡•…”‹„‡†‹–Š‡–‡•–‹–‡ǯ••’‡…‹ϐ‹…ƒ–‹‘Ǥ‡•–‹’—–•ƒ”‡–Š‡•‡Ž‡…–‡†ˆ”‘–Š‡‹’—–’ƒ”–‹–‹‘•Ǥ

ʹ ˜ƒŽ‹† ‘—–’—– ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘• –›’‹…ƒŽŽ› …‘””‡•’‘† –‘ ƒ› ‘—–’—–• –Šƒ– Šƒ˜‡ ‘– „‡‡
‡š’Ž‹…‹–Ž›•’‡…‹ϐ‹‡†Ǥ•–Š‡•‡ƒ”‡‘–•’‡…‹ϐ‹‡†–Š‡‹”‹†‡–‹ϐ‹…ƒ–‹‘‘ˆ–‡”‡•—Ž–•‹‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•„ƒ•‡†
‘ –Š‡ •—„Œ‡…–‹˜‹–› ‘ˆ –Š‡ ‹†‹˜‹†—ƒŽ –‡•–‡”Ǥ Š‹• •—„Œ‡…–‹˜‡ ˆ‘” ‘ˆ –‡•– †‡•‹‰ ƒ› ƒŽ•‘ ‘……—” ™Š‡ ƒ’’Ž›‹‰
experience-based techniques like error guessing.

͵ ‘ƒ‹ ƒƒŽ›•‹• ȋ‡‹œ‡” ͳͻͻͷȌ ‹• ‘ˆ–‡ …Žƒ••‹ϐ‹‡† ƒ• ƒ …‘„‹ƒ–‹‘ ‘ˆ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘‹‰ ƒ†
„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•Ǥ

5.2.1.2 Derive Test Coverage Items (TD3)

ƒ…Š‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ŠƒŽŽ„‡‹†‡–‹ϐ‹‡†ƒ•ƒ–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ‹Ǥ‡Ǥˆ‘”‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰
the test conditions and test coverage items are the same equivalence partitions).

5.2.1.3 Derive Test Cases (TD4)

Test cases shall be derived to exercise the test coverage items (i.e. the equivalence partitions). The
following steps shall be used during test case derivation:
ƒȌ ‡…‹†‡ ‘ ƒ ƒ’’”‘ƒ…Š ˆ‘” •‡Ž‡…–‹‰ …‘„‹ƒ–‹‘• ‘ˆ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• –‘ „‡ ‡š‡”…‹•‡† „› –‡•–
…ƒ•‡•ǡ™Š‡”‡–™‘…‘‘ƒ’’”‘ƒ…Š‡•ƒ”‡ȋ͹ͻʹͷǦʹǣͳͻͻͺǢ›‡”•ͳͻ͹ͻȌǣ
ͳȌ ‘‡Ǧ–‘Ǧ‘‡ǡ‹™Š‹…Š‡ƒ…Š–‡•–…ƒ•‡‹•†‡”‹˜‡†–‘…‘˜‡”ƒ•’‡…‹ϐ‹…‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘Ǣ
ʹȌ ‹‹‹œ‡†ǡ‹™Š‹…Š‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ƒ”‡…‘˜‡”‡†„›–‡•–…ƒ•‡••—…Š–Šƒ––Š‡‹‹—
number of test cases derived covers all equivalence partitions at least once.
 –Š‡”ƒ’’”‘ƒ…Š‡•–‘•‡Ž‡…–‹‰…‘„‹ƒ–‹‘•‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•–‘„‡‡š‡”…‹•‡†„›–‡•–…ƒ•‡•
are described in 5.2.5 (Combinatorial Test Design Techniques).

b) Select test coverage item(s) for inclusion in the current test case based on the approach chosen in
step a);
…Ȍ †‡–‹ˆ›‹’—–˜ƒŽ—‡•–‘‡š‡”…‹•‡–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–‘„‡…‘˜‡”‡†„›–Š‡–‡•–…ƒ•‡ƒ†ƒ”„‹–”ƒ”›
˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†„›–Š‡–‡•–…ƒ•‡Ǣ
†Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
e) Repeat steps b) to d) until the required level of test coverage is achieved.

ͷǤʹǤʹ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†

5.2.2.1 Derive Test Conditions (TD2)

Š‡ …Žƒ••‹ϐ‹…ƒ–‹‘ –”‡‡ ‡–Š‘† ȋ

”‘…Š–ƒ ƒ†
”‹ ͳͻͻ͵Ȍ —•‡• ƒ ‘†‡Ž ‘ˆ –Š‡ –‡•– ‹–‡ –Šƒ–
’ƒ”–‹–‹‘• –Š‡ ‹’—–• ‘ˆ –Š‡ –‡•– ‹–‡ ƒ† ”‡’”‡•‡–• –Š‡ ‰”ƒ’Š‹…ƒŽŽ› ‹ –Š‡ ˆ‘” ‘ˆ ƒ –”‡‡ …ƒŽŽ‡†
ƒ …Žƒ••‹ϐ‹…ƒ–‹‘ –”‡‡Ǥ Š‡ –‡•– ‹–‡ǯ• ‹’—–• ƒ”‡ ’ƒ”–‹–‹‘‡† ‹–‘ Dz…Žƒ••‹ϐ‹…ƒ–‹‘•dzǡ ™Š‡”‡ ‡ƒ…Š
…Žƒ••‹ϐ‹…ƒ–‹‘ …‘•‹•–• ‘ˆ ƒ †‹•Œ‘‹– ȋ‘Ǧ‘˜‡”Žƒ’’‹‰Ȍ •‡– ‘ˆ Dz…Žƒ••‡•dz ƒ† ‘ˆ–‡ •—„Ǧ…Žƒ••‡•ǡ ƒ† –Š‡
•‡–‘ˆ…Žƒ••‹ϐ‹…ƒ–‹‘•‹•…‘’Ž‡–‡ȋƒŽŽ…Žƒ••‹ϐ‹…ƒ–‹‘•‘ˆƒŽŽ‹’—–•”‡Ž‡˜ƒ––‘–Š‡–‡•–‹–‡†‘ƒ‹„‡‹‰
‘†‡ŽŽ‡†Šƒ˜‡„‡‡‹†‡–‹ϐ‹‡†ƒ†‹…Ž—†‡†ȌǤƒ…Š…Žƒ••‹ϐ‹…ƒ–‹‘•ŠƒŽŽ„‡ƒ–‡•–…‘†‹–‹‘ǤDzŽƒ••‡•dz–Šƒ–
”‡•—Ž– ˆ”‘†‡…‘’‘•‹‰ –Š‡…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ›„‡’ƒ”–‹–‹‘‡†ˆ—”–Š‡”‹–‘Dz•—„Ǧ…Žƒ••‡•dz †‡’‡†‹‰
‘–Š‡Ž‡˜‡Ž‘ˆ”‹‰‘—””‡“—‹”‡†‹–Š‡–‡•–‹‰ǤŽƒ••‹ϐ‹…ƒ–‹‘•ƒ†…Žƒ••‡•ƒ›„‡†‡”‹˜‡†ˆ‘”„‘–Š˜ƒŽ‹†

8 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

and invalid input data, depending on the level of test coverage required. The hierarchical relationships
„‡–™‡‡…Žƒ••‹ϐ‹…ƒ–‹‘•ǡ…Žƒ••‡•ƒ†•—„Ǧ…Žƒ••‡•ƒ”‡‘†‡ŽŽ‡†ƒ•ƒ–”‡‡ǡ‹™Š‹…Š–Š‡‹’—–†‘ƒ‹‘ˆ–Š‡
–‡•–‹–‡‹•’Žƒ…‡†ƒ•–Š‡”‘‘–‘†‡ǡ–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ•„”ƒ…Š‘†‡•ǡƒ†–Š‡…Žƒ••‡•‘”•—„Ǧ…Žƒ••‡•
as leaf nodes.
 Š‡ ’”‘…‡•• ‘ˆ ’ƒ”–‹–‹‘‹‰ ‹ –Š‡ …Žƒ••‹ϐ‹…ƒ–‹‘ –”‡‡ ‡–Š‘† ‹• •‹‹Žƒ” –‘ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘‹‰Ǥ
Š‡‡›†‹ˆˆ‡”‡…‡‹•–Šƒ–‹–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†ǡ–Š‡’ƒ”–‹–‹‘•ȋ™Š‹…Šƒ”‡…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ†…Žƒ••‡•Ȍ
—•– „‡ …‘’Ž‡–‡Ž› †‹•Œ‘‹–ǡ ™Š‡”‡ƒ• ‹ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘‹‰ǡ –Š‡› …‘—Ž† ‘˜‡”Žƒ’ †‡’‡†‹‰ ‘ Š‘™ –Š‡
–‡…Š‹“—‡™ƒ•ƒ’’Ž‹‡†Ǥƒ††‹–‹‘ǡ–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†ƒŽ•‘‹…Ž—†‡•–Š‡†‡•‹‰‘ˆƒ…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡ǡ
which provides a visual representation of the test conditions.

5.2.2.2 Derive Test Coverage Items (TD3)

‡•–…‘˜‡”ƒ‰‡‹–‡••ŠƒŽŽ„‡†‡”‹˜‡†„›…‘„‹‹‰…Žƒ••‡•—•‹‰ƒ…Š‘•‡…‘„‹ƒ–‹‘ƒ’’”‘ƒ…ŠǤ
 ™‘‡šƒ’Ž‡ƒ’’”‘ƒ…Š‡•ˆ‘”…‘„‹‹‰…Žƒ••‡•‹–‘–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡ǣ

— minimized, in which classes are included in test coverage items such that the minimum number of test
coverage items are derived to cover all classes at least once;

— maximized, in which classes are included in test coverage items such that each possible combination of
…Žƒ••‡•‹•…‘˜‡”‡†„›ƒ–Ž‡ƒ•–‘‡–‡•–…‘˜‡”ƒ‰‡‹–‡Ǥ

NOTE 1 Other approaches to selecting combinations of test coverage items are described in 5.2.5
(Combinatorial Test Design Techniques).

NOTE 2 The test coverage items are often illustrated in a combination table (see Figure B.5 in B.2.2.5).

͵ Š‡ ‘”‹‰‹ƒŽ ’—„Ž‹…ƒ–‹‘ ‘ˆ –Š‡ …Žƒ••‹ϐ‹…ƒ–‹‘ –”‡‡ ‡–Š‘† ȋ

”‘…Š–ƒ ƒ†
”‹ ͳͻͻ͵Ȍ —•‡† –Š‡
terms “minimal” and “maximal” instead of “minimized” and “maximized”.

5.2.2.3 Derive Test Cases (TD4)

Test cases shall be derived to exercise the test coverage items. The following steps shall be followed
during test case derivation:
a) Based on the combinations of classes created in step TD3, select one combination for inclusion in
–Š‡…—””‡––‡•–…ƒ•‡–Šƒ–Šƒ•‘–ƒŽ”‡ƒ†›„‡‡…‘˜‡”‡†„›ƒ–‡•–…ƒ•‡Ǣ
„Ȍ †‡–‹ˆ›‹’—–˜ƒŽ—‡•ˆ‘”ƒ›…Žƒ••‡•–Šƒ–†‘‘–ƒŽ”‡ƒ†›Šƒ˜‡ƒƒ••‹‰‡†˜ƒŽ—‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.2.3 Boundary Value Analysis

5.2.3.1 Derive Test Conditions (TD2)

‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹• ȋ ͹ͻʹͷǦʹǣͳͻͻͺǢ ›‡”• ͳͻ͹ͻȌ —•‡• ƒ ‘†‡Ž ‘ˆ –Š‡ –‡•– ‹–‡ –Šƒ– ’ƒ”–‹–‹‘•
the inputs and outputs of the test item into a number of ordered sets and subsets (partitions and sub-
’ƒ”–‹–‹‘•Ȍ™‹–Š‹†‡–‹ϐ‹ƒ„Ž‡„‘—†ƒ”‹‡•ǡ™Š‡”‡‡ƒ…Š„‘—†ƒ”›‹•ƒ–‡•–…‘†‹–‹‘ǤŠ‡„‘—†ƒ”‹‡••ŠƒŽŽ
be derived from the test basis.
 ‘”ƒ’ƒ”–‹–‹‘†‡ϐ‹‡†ƒ•‹–‡‰‡”•ˆ”‘ͳ–‘ͳͲ‹…Ž—•‹˜‡ǡ–Š‡”‡ƒ”‡–™‘„‘—†ƒ”‹‡•ǡ™Š‡”‡–Š‡
Ž‘™‡”„‘—†ƒ”›‹•ͳƒ†–Š‡—’’‡”„‘—†ƒ”›‹•ͳͲǡƒ†–Š‡•‡ƒ”‡–Š‡–‡•–…‘†‹–‹‘•Ǥ

NOTE For output boundaries, corresponding input partitions are derived based on the processing described
‹–Š‡–‡•–‹–‡ǯ••’‡…‹ϐ‹…ƒ–‹‘Ǥ‡•–‹’—–•ƒ”‡–Š‡•‡Ž‡…–‡†ˆ”‘–Š‡‹’—–’ƒ”–‹–‹‘•Ǥ

© ISO/IEC 2015 – All rights reserved 9

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.2.3.2 Derive Test Coverage Items (TD3)

One of the following two options for the derivation of test coverage items shall be applied:
Ȅ –™‘Ǧ˜ƒŽ—‡„‘—†ƒ”›–‡•–‹‰Ǣ‘”
Ȅ –Š”‡‡Ǧ˜ƒŽ—‡„‘—†ƒ”›–‡•–‹‰Ǥ
For two-value boundary testingǡ –™‘ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• •ŠƒŽŽ „‡ †‡”‹˜‡† ˆ‘” ‡ƒ…Š „‘—†ƒ”› ȋ–‡•–
…‘†‹–‹‘Ȍ…‘””‡•’‘†‹‰–‘˜ƒŽ—‡•‘–Š‡„‘—†ƒ”›ƒ†ƒ‹…”‡‡–ƒŽ†‹•–ƒ…‡outside–Š‡„‘—†ƒ”›
‘ˆ–Š‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘ǤŠ‹•‹…”‡‡–ƒŽ†‹•–ƒ…‡•ŠƒŽŽ„‡†‡ϐ‹‡†ƒ•–Š‡•ƒŽŽ‡•–•‹‰‹ϐ‹…ƒ–˜ƒŽ—‡
ˆ‘”–Š‡†ƒ–ƒ–›’‡—†‡”…‘•‹†‡”ƒ–‹‘Ǥ
For three-value boundary testingǡ –Š”‡‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• •ŠƒŽŽ „‡ †‡”‹˜‡† ˆ‘” ‡ƒ…Š „‘—†ƒ”› ȋ–‡•–
…‘†‹–‹‘Ȍ …‘””‡•’‘†‹‰ –‘ ˜ƒŽ—‡• ‘ –Š‡ „‘—†ƒ”› ƒ† ƒ ‹…”‡‡–ƒŽ †‹•–ƒ…‡ each side of the
„‘—†ƒ”› ‘ˆ –Š‡ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘Ǥ Š‹• ‹…”‡‡–ƒŽ †‹•–ƒ…‡ •ŠƒŽŽ „‡ †‡ϐ‹‡† ƒ• –Š‡ •ƒŽŽ‡•–
•‹‰‹ϐ‹…ƒ–˜ƒŽ—‡ˆ‘”–Š‡†ƒ–ƒ–›’‡—†‡”…‘•‹†‡”ƒ–‹‘Ǥ
ͳ ‘‡ ’ƒ”–‹–‹‘• …‘—Ž† Šƒ˜‡ ‘Ž› ƒ •‹‰Ž‡ „‘—†ƒ”› ‹†‡–‹ϐ‹‡† ‹ –Š‡ –‡•– „ƒ•‹•Ǥ ‘” ‡šƒ’Ž‡ǡ –Š‡
—‡”‹…ƒŽ’ƒ”–‹–‹‘Dzƒ‰‡η͹Ͳ›‡ƒ”•dzŠƒ•ƒŽ‘™‡”„‘—†ƒ”›„—–‘–ƒ‘„˜‹‘—•—’’‡”„‘—†ƒ”›Ǥ•‘‡…ƒ•‡•ƒ
„‘—†ƒ”›˜ƒŽ—‡‹’‘•‡†„›–Š‡ƒ…–—ƒŽ‹’Ž‡‡–ƒ–‹‘…ƒ„‡—•‡†ƒ•ƒ„‘—†ƒ”›˜ƒŽ—‡ǡ•—…Šƒ•–Š‡Žƒ”‰‡•–˜ƒŽ—‡
–Šƒ–‹•ƒ……‡’–‡†„›–Š‡‹’—–ϐ‹‡Ž†ȋ•—…Š†‡…‹•‹‘••Š‘—Ž†„‡†‘…—‡–‡†Ǣˆ‘”‡šƒ’Ž‡ǡ‹–Š‡‡•–’‡…‹ϐ‹…ƒ–‹‘
documentation).

ʹ ™‘Ǧ˜ƒŽ—‡ „‘—†ƒ”› ˜ƒŽ—‡ –‡•–‹‰ ‹• –›’‹…ƒŽŽ› ƒ†‡“—ƒ–‡ ‹ ‘•– •‹–—ƒ–‹‘•Ǣ Š‘™‡˜‡”ǡ –Š”‡‡Ǧ˜ƒŽ—‡
„‘—†ƒ”›–‡•–‹‰…‘—Ž†„‡”‡“—‹”‡†ˆ‘”…‡”–ƒ‹…‹”…—•–ƒ…‡•ȋ‡Ǥ‰Ǥˆ‘””‹‰‘”‘—•–‡•–‹‰–‘…Š‡…–Šƒ–‘‡””‘”•
™‡”‡ƒ†‡‹†‡–‡”‹‹‰–Š‡„‘—†ƒ”‹‡•‘ˆ˜ƒ”‹ƒ„Ž‡•‹–Š‡–‡•–‹–‡„›„‘–Š–‡•–‡”•ƒ††‡˜‡Ž‘’‡”•ȌǤ

͵ „‘–Š–™‘Ǧƒ†–Š”‡‡Ǧ˜ƒŽ—‡„‘—†ƒ”›˜ƒŽ—‡–‡•–‹‰ǡ…‘–‹‰—‘—•’ƒ”–‹–‹‘•ȋ’ƒ”–‹–‹‘•–Šƒ–•Šƒ”‡ƒ
„‘—†ƒ”›Ȍ™‹ŽŽ”‡•—Ž–‹†—’Ž‹…ƒ–‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ǡ‹™Š‹…Š…ƒ•‡‹–‹•–›’‹…ƒŽ’”ƒ…–‹…‡–‘‘Ž›‡š‡”…‹•‡–Š‡•‡
duplicated values once. For an example of duplicate boundaries, see B.2.3.4.3.

5.2.3.3 Derive Test Cases (TD4)

Test cases shall be derived to exercise the test coverage items. The following steps shall be used during
test case derivation:
ƒȌ ‡…‹†‡ ‘ ƒ ƒ’’”‘ƒ…Š ˆ‘” •‡Ž‡…–‹‰ …‘„‹ƒ–‹‘• ‘ˆ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• –‘ „‡ ‡š‡”…‹•‡† „› –‡•–
…ƒ•‡•ǡ™Š‡”‡–™‘…‘‘ƒ’’”‘ƒ…Š‡•ƒ”‡ȋ͹ͻʹͷǦʹǣͳͻͻͺǢ›‡”•ͳͻ͹ͻȌǣ
ͳȌ ‘‡Ǧ–‘Ǧ‘‡ǡ‹™Š‹…Š‡ƒ…Š–‡•–…ƒ•‡‹•†‡”‹˜‡†–‘‡š‡”…‹•‡ƒ•’‡…‹ϐ‹…„‘—†ƒ”›˜ƒŽ—‡Ǣ
ʹȌ ‹‹‹œ‡†ǡ‹™Š‹…Š„‘—†ƒ”›˜ƒŽ—‡•ƒ”‡‹…Ž—†‡†‹–‡•–…ƒ•‡••—…Š–Šƒ––Š‡‹‹——„‡”
‘ˆ–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†–‘…‘˜‡”ƒŽŽ„‘—†ƒ”›˜ƒŽ—‡•ƒ–Ž‡ƒ•–‘…‡Ǥ
ͳ ‹‹‹œ‡†„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ǡ‡ƒ…Š–‡•–…ƒ•‡…ƒ…‘˜‡”—Ž–‹’Ž‡–‡•–…‘˜‡”ƒ‰‡‹–‡•Ǥ

ʹ –Š‡”ƒ’’”‘ƒ…Š‡•–‘•‡Ž‡…–‹‰…‘„‹ƒ–‹‘•‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•–‘„‡‡š‡”…‹•‡†„›–‡•–…ƒ•‡•
are described in 5.2.5 (Combinatorial Test Design Techniques).

b) Select test coverage item(s) for inclusion in the current test case based on the approach chosen in
step a);
…Ȍ †‡–‹ˆ›ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†„›–Š‡–‡•–…ƒ•‡–Šƒ–™‡”‡‘–
ƒŽ”‡ƒ†›•‡Ž‡…–‡†‹•–‡’„ȌǢ
†Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
e) Repeat steps b) to d) until the required level of test coverage is achieved.

10 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

5.2.4 Syntax Testing

5.2.4.1 Derive Test Conditions (TD2)

›–ƒš–‡•–‹‰ȋ‡‹œ‡”ͳͻͻͷǢ—”•–‡‹ʹͲͲ͵Ȍ—•‡•ƒˆ‘”ƒŽ‘†‡Ž‘ˆ–Š‡‹’—–•–‘ƒ–‡•–‹–‡ƒ•–Š‡
„ƒ•‹•ˆ‘”–‡•–†‡•‹‰ǤŠ‹••›–ƒš‘†‡Ž‹•”‡’”‡•‡–‡†ƒ•ƒ—„‡”‘ˆ”—Ž‡•ǡ™Š‡”‡‡ƒ…Š”—Ž‡†‡ϐ‹‡•
the format of an input parameter in terms of “sequences of”, “iterations of”, or “selections between”
‡Ž‡‡–•‹–Š‡•›–ƒšǤŠ‡•›–ƒšƒ›„‡”‡’”‡•‡–‡†‹ƒ–‡š–—ƒŽ‘”†‹ƒ‰”ƒƒ–‹…ˆ‘”ƒ–ǤŠ‡–‡•–
…‘†‹–‹‘‹•›–ƒš–‡•–‹‰•ŠƒŽŽ„‡–Š‡™Š‘Ž‡‘”’ƒ”–‹ƒŽ‘†‡Ž‘ˆ–Š‡‹’—–•–‘–Š‡–‡•–‹–‡Ǥ
ͳ ƒ…—•Ǧƒ—” ‘”‹•ƒˆ‘”ƒŽ‡–ƒǦŽƒ‰—ƒ‰‡–Šƒ–…ƒ„‡—•‡†–‘†‡ϐ‹‡–Š‡•›–ƒšˆ‘”ƒ–‡•–‹–‡
in a textual format.

ʹ ƒ„•–”ƒ…–•›–ƒš–”‡‡…ƒ„‡—•‡†–‘”‡’”‡•‡–ˆ‘”ƒŽ•›–ƒš†‹ƒ‰”ƒƒ–‹…ƒŽŽ›Ǥ

5.2.4.2 Derive Test Coverage Items (TD3)

•›–ƒš–‡•–‹‰–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡†‡”‹˜‡†„ƒ•‡†‘–™‘‰‘ƒŽ•ǣ’‘•‹–‹˜‡–‡•–‹‰ǡ‹™Š‹…Š–‡•–
…‘˜‡”ƒ‰‡‹–‡•ƒ”‡†‡”‹˜‡†–‘…‘˜‡”–Š‡˜ƒŽ‹†•›–ƒš‹˜ƒ”‹‘—•™ƒ›•ǡƒ†‡‰ƒ–‹˜‡–‡•–‹‰ǡ‹™Š‹…Š
–‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ †‡”‹˜‡† –‘ †‡Ž‹„‡”ƒ–‡Ž› ˜‹‘Žƒ–‡ –Š‡ ”—Ž‡• ‘ˆ –Š‡ •›–ƒšǤ ‡•– …‘˜‡”ƒ‰‡ ‹–‡•
ˆ‘”’‘•‹–‹˜‡–‡•–‹‰•ŠƒŽŽ„‡Dz‘’–‹‘•dz‘ˆ–Š‡†‡ϐ‹‡†•›–ƒšǡ™Š‡”‡ƒ•–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”‡‰ƒ–‹˜‡
–‡•–‹‰•ŠƒŽŽ„‡Dz—–ƒ–‹‘•dz‘ˆ–Š‡†‡ϐ‹‡†•›–ƒšǤ
Š‡ˆ‘ŽŽ‘™‹‰‰—‹†‡Ž‹‡•ƒ›„‡—•‡†–‘†‡”‹˜‡Dz‘’–‹‘•dzȋƒŽ–Š‘—‰ŠƒŽ–‡”ƒ–‹˜‡‰—‹†‡Ž‹‡•ƒ›„‡—•‡†
where appropriate):
Ȅ ™Š‡‡˜‡”•‡Ž‡…–‹‘‹•ƒ†ƒ–‡†„›–Š‡•›–ƒšǡƒDz‘’–‹‘dz‹•†‡”‹˜‡†ˆ‘”‡ƒ…ŠƒŽ–‡”ƒ–‹˜‡’”‘˜‹†‡†
for that selection;
ͳ ‘” –Š‡ ‹’—– ’ƒ”ƒ‡–‡” Dz…‘Ž‘—” α Ž—‡ ȁ ‡† ȁ
”‡‡dz ȋ™Š‡”‡ ȁ ”‡’”‡•‡–• –Š‡ Dzdz ‘‘Ž‡ƒ
operator), three options “Blue”, “Red” and “Green” will be derived as test coverage items.

Ȅ ™Š‡‡˜‡”ƒ‹–‡”ƒ–‹‘‹•ƒ†ƒ–‡†„›–Š‡•›–ƒšǡƒ–Ž‡ƒ•––™‘Dz‘’–‹‘•dzƒ”‡†‡”‹˜‡†ˆ‘”–Š‡‹–‡”ƒ–‹‘Ǣ
one with the minimum number of repetitions and the other with more than the minimum number
of repetitions;
ʹ ‘”–Š‡‹’—–’ƒ”ƒ‡–‡”DzŽ‡––‡”αȏȂȁƒȂœȐ+” (where “+” represents “one or more”), two options,
“one letter” and “more than one letter”, will be derived as test coverage items.

— whenever iteration is mandated with a maximum number of repetitions, at least two “options” are
derived for the iteration; one with the maximum number of repetitions and the other with more
than the maximum number of repetitions.
͵ ‘”–Š‡‹’—–’ƒ”ƒ‡–‡”DzŽ‡––‡”αȏȂȁƒȂœȐ100” (where “100” represents the fact that a letter
can be chosen up to 100 times), two options “100 letters” and “more than 100 letters” will be derived as test
coverage items.

Š‡ ˆ‘ŽŽ‘™‹‰ ‰—‹†‡Ž‹‡ ƒ› „‡ —•‡† –‘ †‡”‹˜‡ Dz—–ƒ–‹‘•dz ȋƒŽ–Š‘—‰Š ƒŽ–‡”ƒ–‹˜‡ ‰—‹†‡Ž‹‡• ƒ› „‡
used where appropriate):
Ȅ ˆ‘”ƒ›‹’—–ǡ–Š‡†‡ϐ‹‡†•›–ƒšƒ›„‡—–ƒ–‡†–‘†‡”‹˜‡‹˜ƒŽ‹†‹’—–•ȋDz—–ƒ–‹‘•dzȌǤ
Ͷ ‘”–Š‡‹’—–’ƒ”ƒ‡–‡”Dz…‘Ž‘—”αŽ—‡ȁ‡†ȁ
”‡‡dzǡ‘‡Dz—–ƒ–‹‘dz…‘—Ž†„‡–‘‹–”‘†—…‡ƒ
invalid value for the parameter as the test coverage item, such as selecting the value “Yellow”, which does not
appear in the input parameter list. Other example mutations are provided in B.2.4.5.

© ISO/IEC 2015 – All rights reserved 11

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.2.4.3 Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘”•›–ƒš–‡•–‹‰•ŠƒŽŽ„‡†‡”‹˜‡†–‘…‘˜‡”–Š‡…Š‘•‡‘’–‹‘•ƒ†—–ƒ–‹‘•ǤŠ‡ˆ‘ŽŽ‘™‹‰
steps shall be used during test case derivation:
ƒȌ ‡…‹†‡ ‘ ƒ ƒ’’”‘ƒ…Š ˆ‘” •‡Ž‡…–‹‰ …‘„‹ƒ–‹‘• ‘ˆ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• –‘ „‡ ‡š‡”…‹•‡† „› –‡•–
cases, where two common approaches are:
ͳȌ ‘‡Ǧ–‘Ǧ‘‡ǡ ‹ ™Š‹…Š ‡ƒ…Š –‡•– …ƒ•‡ ‹• †‡”‹˜‡† –‘ ‡š‡”…‹•‡ ƒ •’‡…‹ϐ‹… ‘’–‹‘ǡ ‹–‡”ƒ–‹‘ ƒ†Ȁ‘”
mutation; or
2) minimized, in which options, iterations and/or mutations are included in test cases such that
the minimum number of test cases are derived to cover all options, iterations and/or mutations
at least once.
 –Š‡”ƒ’’”‘ƒ…Š‡•–‘•‡Ž‡…–‹‰…‘„‹ƒ–‹‘•‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•–‘„‡‡š‡”…‹•‡†„›–‡•–…ƒ•‡•
are described in 5.2.5 (Combinatorial Test Design Techniques).

b) Select test coverage item(s) for inclusion in the current test case;
…Ȍ †‡–‹ˆ› ‹’—– ˜ƒŽ—‡• –‘ ‡š‡”…‹•‡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡ȋ•Ȍ –‘ „‡ …‘˜‡”‡† „› –Š‡ –‡•– …ƒ•‡ ƒ†
ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†„›–Š‡–‡•–…ƒ•‡Ǣ
†Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
e) Repeat steps b) to d) until all derived options, iterations and/or mutations are exercised.

5.2.5 Combinatorial Test Design Techniques

5.2.5.1 Overview

‘„‹ƒ–‘”‹ƒŽ–‡•–†‡•‹‰–‡…Š‹“—‡•ƒ”‡—•‡†–‘•›•–‡ƒ–‹…ƒŽŽ›†‡”‹˜‡ƒ‡ƒ‹‰ˆ—Žƒ†ƒƒ‰‡ƒ„Ž‡
subset of test cases that cover the test conditions and test coverage items that are derivable during
–‡•–‹‰Ǥ Š‡ …‘„‹ƒ–‹‘• ‘ˆ ‹–‡”‡•– ƒ”‡ †‡ϐ‹‡† ‹ –‡”• ‘ˆ –‡•– ‹–‡ ’ƒ”ƒ‡–‡”• ƒ† –Š‡ ˜ƒŽ—‡•
these parameters can take. Where numerous parameters (each with numerous discrete values) must
‹–‡”ƒ…–ǡ –Š‹• –‡…Š‹“—‡ ‡ƒ„Ž‡• ƒ •‹‰‹ϐ‹…ƒ– ”‡†—…–‹‘ ‹ –Š‡ —„‡” ‘ˆ –‡•– …ƒ•‡• ”‡“—‹”‡† ™‹–Š‘—–
compromising functional coverage.

5.2.5.2 Derive Test Conditions (TD2)

The test item parameters represent particular aspects of the test item that are relevant to the testing,
ƒ†‘ˆ–‡…‘””‡•’‘†–‘–Š‡‹’—–’ƒ”ƒ‡–‡”•–‘–Š‡–‡•–‹–‡ǡ„—–‘–Š‡”ƒ•’‡…–•ƒ›ƒŽ•‘„‡—•‡†Ǥ
ͳ …‘ϐ‹‰—”ƒ–‹‘–‡•–‹‰ǡ–Š‡’ƒ”ƒ‡–‡”•…‘—Ž†„‡˜ƒ”‹‘—•‡˜‹”‘‡–ˆƒ…–‘”•ǡ•—…Šƒ•‘’‡”ƒ–‹‰
•›•–‡ƒ†„”‘™•‡”Ǥ

Each test item parameter can take on various values. For use in this technique the set of values needs
–‘„‡ϐ‹‹–‡ƒ†ƒƒ‰‡ƒ„Ž‡Ǥ‘‡–‡•–‹–‡’ƒ”ƒ‡–‡”•ƒ›„‡ƒ–—”ƒŽŽ›…‘•–”ƒ‹‡†–‘‘Ž›–ƒ‡‘ƒ
•ƒŽŽ•‡–‘ˆ’‘••‹„Ž‡˜ƒŽ—‡•™Š‹Ž‡‘–Š‡”–‡•–‹–‡’ƒ”ƒ‡–‡”•ƒ›„‡ˆƒ”Ž‡••…‘•–”ƒ‹‡†Ǥ
ʹ –‡•–‹–‡’ƒ”ƒ‡–‡”–Šƒ–‹•ƒ–—”ƒŽŽ›…‘•–”ƒ‹‡†–‘ƒ•ƒŽŽ—„‡”‘ˆ˜ƒŽ—‡•‹•ƒDz†ƒ›̴‘ˆ̴™‡‡dz
’ƒ”ƒ‡–‡” –Šƒ– …ƒ ‘Ž› –ƒ‡ ‘ •‡˜‡ •’‡…‹ϐ‹… ˜ƒŽ—‡• Dzȏ‘†ƒ› ȁ —‡•†ƒ› ȁ ‡†‡•†ƒ› ȁ Š—”•†ƒ› ȁ ”‹†ƒ› ȁ
ƒ–—”†ƒ›ȁ—†ƒ›ȐdzǤ

͵ –‡•–‹–‡’ƒ”ƒ‡–‡”–Šƒ–‹•ˆƒ”Ž‡••…‘•–”ƒ‹‡†‹•ƒ’ƒ”ƒ‡–‡”…‘•‹•–‹‰‘ˆƒ›‡ƒŽ—„‡”ǡ
™Š‹…Š‡š‹•–•ƒŽ‘‰ƒ’‘–‡–‹ƒŽŽ›‹ϐ‹‹–‡—„‡”Ž‹‡Ǥ

‘”—…‘•–”ƒ‹‡†–‡•–‹–‡’ƒ”ƒ‡–‡”•ǡ‹–ƒ›ϐ‹”•–„‡‡…‡••ƒ”›–‘ƒ’’Ž›‘–Š‡”–‡•–†‡•‹‰–‡…Š‹“—‡•ǡ
•—…Šƒ•‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰‘”„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ǡ–‘”‡†—…‡ƒŽƒ”‰‡•‡–‘ˆ’‘••‹„Ž‡˜ƒŽ—‡•ˆ‘”
a parameter to a manageable subset.

12 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

The test conditions for combinatorial testing are the same for all the combinatorial test design
–‡…Š‹“—‡•Ǣ‡ƒ…Š–‡•–…‘†‹–‹‘•ŠƒŽŽ„‡ƒ…Š‘•‡–‡•–‹–‡’ƒ”ƒ‡–‡”ȋȌ–ƒ‹‰‘ƒ•’‡…‹ϐ‹…˜ƒŽ—‡ȋȌǡ
resulting in a P-V pair.

5.2.5.3 All Combinations Testing

5.2.5.3.1 Derive Test Coverage Items (TD3)

ǮƒŽŽ…‘„‹ƒ–‹‘•ǯ–‡•–‹‰ȋ
”‹†ƒŽǡˆˆ—––ƒ††Ž‡”ʹͲͲͷȌ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡••ŠƒŽŽ„‡‡„‡”•‘ˆ
the set of all unique combinations of P-V pairs, such that each parameter is included at least once in the set.

5.2.5.3.2 Derive Test Cases (TD4)

Test cases shall be derived in which each test case exercises one unique combination of P-V pairs. The
following steps shall be used during test case derivation:
ƒȌ ‡Ž‡…––‡•–…‘˜‡”ƒ‰‡‹–‡ȋ•Ȍˆ‘”‹…Ž—•‹‘‹–Š‡…—””‡––‡•–…ƒ•‡–Šƒ–Šƒ˜‡‘–ƒŽ”‡ƒ†›„‡‡…‘˜‡”‡†
„›ƒ–‡•–…ƒ•‡Ǣ
„Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
c) Repeat steps a) and b) until the required level of test coverage is achieved.
NOTE The minimum number of test cases required to achieve 100% all combinations testing coverage
corresponds to the product of the number of P-V pairs for each test item parameter.

5.2.5.4 Pair-wise Testing

5.2.5.4.1 Derive Test Coverage Items (TD3)

In pair-wise testing (Grindal, Offutt and Andler 2005), the test coverage items shall be unique pairs
of P-V pairs, where each P-V pair within the pair is for a different test item parameter. Instead of all
possible combinations of the parameters (as was required for all combinations testing), this technique
…‘˜‡”•ƒŽŽ’‘••‹„Ž‡’ƒ‹”•‘ˆ–Š‡•‡Ž‡…–‡†˜ƒŽ—‡•™‹–Š‹–Š‡–‘–ƒŽ•‡–ǡ–Š‡”‡„›‡ˆϐ‹…‹‡–Ž›‡š‡”…‹•‹‰–Š‡–‡•–
item with fewer test cases. Pair-wise testing is also known as “all pairs” testing.

5.2.5.4.2 Derive Test Cases (TD4)

ƒ˜‹‰ϐ‹”•–‹†‡–‹ϐ‹‡†–Š‡Ǧ’ƒ‹”•ǡ–‡•–…ƒ•‡••ŠƒŽŽ„‡†‡”‹˜‡†–‘‡š‡”…‹•‡’ƒ‹”•‘ˆǦ’ƒ‹”•ǡ™Š‡”‡‡ƒ…Š
test case exercises one or more unique pairs. The following steps shall be used during test case derivation:
a) Select test coverage item(s) for inclusion in the current test case, where each pair of P-V pairs covers
ƒ†‹ˆˆ‡”‡–’ƒ‹”‘ˆ’ƒ”ƒ‡–‡”˜ƒŽ—‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡‹…Ž—†‡†‹ƒ–‡•–…ƒ•‡Ǣ
„Ȍ †‡–‹ˆ›ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”’ƒ”ƒ‡–‡”•’”‡•‡–‹–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps a) to c) until all unique pairs of P-V pairs have been exercised.
Š‡‹‹——„‡”‘ˆ–‡•–…ƒ•‡•”‡“—‹”‡†–‘ƒ…Š‹‡˜‡ͳͲͲΨ’ƒ‹”Ǧ™‹•‡–‡•–‹‰‹•‘–‡ƒ•‹Ž›…ƒŽ…—Žƒ–‡†Ǥ
 ‡ƒ”Ǧ‘’–‹ƒŽ •‡– ƒ› „‡ …‘•‹†‡”‡† ƒ……‡’–ƒ„Ž‡ ƒ† ƒ› „‡ …ƒŽ…—Žƒ–‡† —•‹‰ ‘‡ ‘ˆ –Š‡ ˆ‘ŽŽ‘™‹‰
three options:
Ȅ ƒ—ƒŽŽ›†‡–‡”‹‡ƒ‡ƒ”Ǧ‘’–‹ƒŽ•‡–—•‹‰ƒƒŽ‰‘”‹–ŠǢ
— use an automated tool (implementing an algorithm) to determine a near-optimal set; or
Ȅ —•‡‘”–Š‘‰‘ƒŽƒ””ƒ›•ȋƒ†ŽͳͻͺͷȌ–‘†‡–‡”‹‡ƒ‡ƒ”Ǧ‘’–‹ƒŽ•‡–Ǥ

© ISO/IEC 2015 – All rights reserved 13

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.2.5.5 Each Choice Testing

5.2.5.5.1 Derive Test Coverage Items (TD3)

In each choice (or 1-wise) testing (Grindal, Offutt and Andler 2005), the test coverage items shall be
members of the set of P-V pairs such that each parameter value is included at least once in the set.

5.2.5.5.2 Derive Test Cases (TD4)

Test cases shall be derived to exercise P-V pairs, where each test case exercises one or more P-V pairs
–Šƒ– Šƒ˜‡ ‘– „‡‡ ’”‡˜‹‘—•Ž› ‹…Ž—†‡† ‹ ƒ –‡•– …ƒ•‡Ǥ Š‡ ˆ‘ŽŽ‘™‹‰ •–‡’• •ŠƒŽŽ „‡ —•‡† †—”‹‰ –‡•–
case derivation:
a) Select test coverage item(s) for inclusion in the current test case, where at least one selected test
coverage item has not been included in a prior test case;
„Ȍ †‡–‹ˆ›ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”’ƒ”ƒ‡–‡”•’”‡•‡–‹–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps a) to c) until the required level of test coverage is achieved.
The minimum number of test cases required to achieve 100% each choice testing corresponds to the
ƒš‹——„‡”‘ˆ˜ƒŽ—‡•ƒ›‘‡‘ˆ–Š‡–‡•–‹–‡’ƒ”ƒ‡–‡”•…ƒ–ƒ‡Ǥ

5.2.5.6 Base Choice Testing

5.2.5.6.1 Derive Test Coverage Items (TD3)

In base choice testing (Grindal, Offutt and Andler 2005), the test coverage items shall be sets of P-V
pairs for each of the input parameters, where all parameters except one are set to their “base” value and
–Š‡ϐ‹ƒŽ’ƒ”ƒ‡–‡”‹••‡––‘‘‡‘ˆ‹–•‘–Š‡”˜ƒŽ‹†˜ƒŽ—‡•Ǥ
 Š‡”‡ƒ”‡ƒ—„‡”‘ˆƒ’’”‘ƒ…Š‡•ˆ‘”…Š‘‘•‹‰–Š‡„ƒ•‡˜ƒŽ—‡•ˆ‘”‡ƒ…Š’ƒ”ƒ‡–‡”Ǥ ‘”‡šƒ’Ž‡ǡ–Š‡›…ƒ
„‡…Š‘•‡ˆ”‘–Š‡‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡ǡˆ”‘–Š‡–›’‹…ƒŽ’ƒ–Š‹•…‡ƒ”‹‘–‡•–‹‰ǡˆ”‘–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–
ƒ”‡†‡”‹˜‡††—”‹‰‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰‘”ˆ”‘–Š‡†‡ˆƒ—Ž–ȋ‘•–ˆ”‡“—‡–Ž›—•‡†Ȍ˜ƒŽ—‡•ˆ‘”–Š‡’ƒ”ƒ‡–‡”Ǥ

5.2.5.6.2 Derive Test Cases (TD4)

ƒ˜‹‰ϐ‹”•–‹†‡–‹ϐ‹‡†–Š‡Ǧ’ƒ‹”•ǡ„ƒ•‡…Š‘‹…‡•ˆ‘”‡ƒ…Š‘ˆ–Š‡’ƒ”ƒ‡–‡”••ŠƒŽŽ„‡…Š‘•‡Ǥ‡•–…ƒ•‡•
•ŠƒŽŽ„‡†‡”‹˜‡†„›•‡––‹‰ƒŽŽ„—–‘‡’ƒ”ƒ‡–‡”–‘‹–•„ƒ•‡…Š‘‹…‡ƒ†–Š‡•‡––‹‰–Š‡ϐ‹ƒŽ’ƒ”ƒ‡–‡”
to a valid value until the required level of test coverage of PV-pairs is achieved. The following steps shall
be used during test case derivation:
ƒȌ ‡”‹˜‡–Š‡„ƒ•‡Ǧ…Š‘‹…‡–‡•–…ƒ•‡„›•‡––‹‰‡ƒ…Š’ƒ”ƒ‡–‡”–‘‹–•Dz„ƒ•‡dz˜ƒŽ—‡Ǣ
„Ȍ ”‡ƒ–‡ƒ‡™–‡•–…ƒ•‡„›•‡––‹‰‘‡’ƒ”ƒ‡–‡”–‘ƒ˜ƒŽ‹†ȋ‘Ǧ„ƒ•‡…Š‘‹…‡Ȍ˜ƒŽ—‡™Š‹Ž‡‡‡’‹‰
the remaining set of parameters to their base choice values;
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡‡™–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–•–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps b) and c) until the required level of test coverage is achieved.

14 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

5.2.6 Decision Table Testing

5.2.6.1 Derive Test Conditions (TD2)

‡…‹•‹‘–ƒ„Ž‡–‡•–‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ›‡”•ͳͻ͹ͻȌ—•‡•ƒ‘†‡Ž‘ˆ–Š‡Ž‘‰‹…ƒŽ”‡Žƒ–‹‘•Š‹’•ȋ†‡…‹•‹‘
rules) between conditions (causes) and actions (effects) for the test item in the form of a decision
table, where:
Ȅ ‡ƒ…Š ‘‘Ž‡ƒ …‘†‹–‹‘ †‡ϐ‹‡• ƒ ’ƒ‹” ‘ˆ ‹’—– ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘• ˆ‘” –Š‡ –‡•– ‹–‡ǡ ‘‡
corresponding to the “true” case, and one to the “false” case;
— each action is an expected outcome or a combination of outcomes for the test item expressed as a
Boolean;
Ȅ ƒ•‡–‘ˆ†‡…‹•‹‘”—Ž‡•†‡ϐ‹‡•–Š‡”‡“—‹”‡†”‡Žƒ–‹‘•Š‹’•„‡–™‡‡…‘†‹–‹‘•ƒ†ƒ…–‹‘•Ǥ
The test conditions shall be the conditions and actions.
NOTE If conditions consist of multiple values rather than simple Booleans, this results in an “extended
‡–”›dz†‡…‹•‹‘–ƒ„Ž‡ǡ–Š‡–‡•–‹‰‘ˆ™Š‹…Š…ƒ„‡Šƒ†Ž‡†„›‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰Ǥ

5.2.6.2 Derive Test Coverage Items (TD3)

 †‡…‹•‹‘ –ƒ„Ž‡ –‡•–‹‰ǡ ‡ƒ…Š †‡…‹•‹‘ ”—Ž‡ǡ ™Š‹…Š †‡ϐ‹‡• –Š‡ ”‡Žƒ–‹‘•Š‹’ „‡–™‡‡ ƒ —‹“—‡
combination of the test item’s conditions and actions, is a test coverage item.

5.2.6.3 Derive Test Cases (TD4)

Test cases shall be derived to exercise the decision rules (test coverage items), where each test case
†‡ϐ‹‡•–Š‡”‡Žƒ–‹‘•Š‹’„‡–™‡‡–Š‡‹’—–•ƒ†‘—–’—–•ǡƒ†‡ƒ…Š†‡…‹•‹‘”—Ž‡…‘””‡•’‘†•–‘ƒ—‹“—‡
combination of Boolean conditions. The following steps shall be used during test case derivation:
a) Select test coverage item(s) from the decision table for implementation as a test case;
„Ȍ †‡–‹ˆ›‹’—–˜ƒŽ—‡•–‘•ƒ–‹•ˆ›–Š‡‹’—–…‘†‹–‹‘ȋ•Ȍ‘ˆ–Š‡†‡…‹•‹‘”—Ž‡ȋ•Ȍ–‘„‡…‘˜‡”‡†„›–Š‡
–‡•–…ƒ•‡ƒ†ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†–‘‡š‡…—–‡–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡†‡…‹•‹‘–ƒ„Ž‡Ǣ
d) Repeat steps a) to c) until the required level of test coverage is achieved.
NOTE If the decision table contains dependent input conditions then this could result in infeasible
combinations (e.g. “age less than 18” and “age greater than 65” both set to true). In this situation such infeasible
†‡…‹•‹‘”—Ž‡••Š‘—Ž†„‡‹†‡–‹ϐ‹‡†ƒ††‘…—‡–‡†ƒ†ƒ”‡‘–—•‡†–‘†‡”‹˜‡–‡•–…ƒ•‡•Ǥ

5.2.7 Cause-Effect Graphing

5.2.7.1 Derive Test Conditions (TD2)

ƒ—•‡Ǧ‡ˆˆ‡…–‰”ƒ’Š‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ›‡”•ͳͻ͹ͻǡ—”•‹—Ž—ƒ†”‘„‡”–ͳͻͻͷȌ—•‡•ƒ‘†‡Ž‘ˆ–Š‡
logical relationships (decision rules) between causes (e.g. inputs) and effects (e.g. outputs) for the test
item in the form of a cause-effect graph, where:
Ȅ ‡ƒ…Š‘‘Ž‡ƒ…ƒ—•‡†‡ϐ‹‡•ƒ’ƒ‹”‘ˆ‹’—–‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ˆ‘”–Š‡–‡•–‹–‡ǡ‘‡…‘””‡•’‘†‹‰
to the “true” case, and one to the “false” case; and
Ȅ ‡ƒ…Š‡ˆˆ‡…–†‡ϐ‹‡•ƒ‡š’‡…–‡†‘—–’—–…‘†‹–‹‘‘”…‘„‹ƒ–‹‘‘ˆ‘—–’—–…‘†‹–‹‘•ˆ‘”–Š‡–‡•–
item expressed as a Boolean.
The test conditions shall be the causes and effects.

© ISO/IEC 2015 – All rights reserved 15

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

The cause-effect graph models logical relationships between causes and effects as a Boolean logic
‡–™‘”™‡‹‰Š–‡†„›‘‘Ž‡ƒ‘’‡”ƒ–‘”•ǡƒ†…ƒ‘’–‹‘ƒŽŽ›‘†‡Ž•›–ƒ…–‹…ƒ†•‡ƒ–‹……‘•–”ƒ‹–•
over relationships between causes and relationships between effects (see Figures B.11 and B.12 in
Annex B.2.7.4).

5.2.7.2 Derive Test Coverage Items (TD3)

 …ƒ—•‡Ǧ‡ˆˆ‡…– ‰”ƒ’Š‹‰ǡ ‡ƒ…Š †‡…‹•‹‘ ”—Ž‡ǡ ™Š‹…Š †‡ϐ‹‡• –Š‡ ”‡Žƒ–‹‘•Š‹’ „‡–™‡‡ ƒ —‹“—‡
combination of the test item’s causes and effects, is a test coverage item.

5.2.7.3 Derive Test Cases (TD4)

‡•–…ƒ•‡••ŠƒŽŽ„‡†‡”‹˜‡†–‘‡š‡”…‹•‡–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•Ǥ…‘””‡•’‘†‹‰†‡…‹•‹‘–ƒ„Ž‡ƒ›„‡
produced from the cause-effect graph and used to derive the test cases. The following steps shall be
used during test case derivation:
a) Select test coverage item(s) to be implemented in the current test case;
„Ȍ †‡–‹ˆ› ‹’—– ˜ƒŽ—‡• –‘ ‡š‡”…‹•‡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡ȋ•Ȍ –‘ „‡ …‘˜‡”‡† „› –Š‡ –‡•– …ƒ•‡ ƒ†
ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†–‘‡š‡…—–‡–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡…ƒ—•‡Ǧ‡ˆˆ‡…–‰”ƒ’Š
and/or decision table;
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.2.8 State Transition Testing

5.2.8.1 Derive Test Conditions (TD2)

–ƒ–‡–”ƒ•‹–‹‘–‡•–‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ‘’‡Žƒ†ʹͲͲͶȌ—•‡•ƒ‘†‡Ž‘ˆ–Š‡•–ƒ–‡•–Š‡–‡•–‹–‡ƒ›
‘……—’›ǡ –Š‡ –”ƒ•‹–‹‘• „‡–™‡‡ •–ƒ–‡•ǡ –Š‡ ‡˜‡–• ™Š‹…Š …ƒ—•‡ –”ƒ•‹–‹‘• ƒ† –Š‡ ƒ…–‹‘• –Šƒ– ƒ›
”‡•—Ž–ˆ”‘–Š‡–”ƒ•‹–‹‘•ǤŠ‡•–ƒ–‡•‘ˆ–Š‡‘†‡Ž•ŠƒŽŽ„‡†‹•…”‡–‡ǡ‹†‡–‹ϐ‹ƒ„Ž‡ƒ†ϐ‹‹–‡‹—„‡”Ǥ
‹†‹˜‹†—ƒŽ–”ƒ•‹–‹‘ƒ›„‡…‘•–”ƒ‹‡†„›ƒ‡˜‡–‰—ƒ”†ǡ™Š‹…Š†‡ϐ‹‡•ƒ•‡–‘ˆ…‘†‹–‹‘•–Šƒ–
must be true when the event occurs, in order for the transition to occur. In state transition testing, the
–‡•–…‘†‹–‹‘•ƒ›„‡ƒŽŽ•–ƒ–‡•‘ˆ–Š‡•–ƒ–‡‘†‡ŽǡƒŽŽ–”ƒ•‹–‹‘•‘ˆ–Š‡•–ƒ–‡‘†‡Ž‘”–Š‡‡–‹”‡•–ƒ–‡
‘†‡Žǡ†‡’‡†‹‰‘–Š‡…‘˜‡”ƒ‰‡”‡“—‹”‡‡–•‘ˆ–‡•–‹‰ǤŠ‡‘†‡Žƒ›„‡”‡’”‡•‡–‡†ƒ•ƒ•–ƒ–‡
–”ƒ•‹–‹‘†‹ƒ‰”ƒ‘”ƒ•–ƒ–‡–ƒ„Ž‡ȋƒŽ–Š‘—‰Š‘–Š‡””‡’”‡•‡–ƒ–‹‘•ƒ›ƒŽ•‘„‡—•‡†ȌǤ

5.2.8.2 Derive Test Coverage Items (TD3)

In state transition testing, test coverage items will change depending on the chosen test completion
criterion and test design approach. Possible test completion criteria include but are not limited to
the following:
— states, in which test coverage items shall be derived to enable all states in the state model to be
“visited”;
— single transitions (0-switch coverage), in which test coverage items shall be derived to cover valid
single transitions in the state model;
— all transitions, in which test coverage items shall be derived to cover both valid transitions in the
•–ƒ–‡‘†‡Žƒ†Dz‹˜ƒŽ‹†dz–”ƒ•‹–‹‘•ȋ–”ƒ•‹–‹‘•ˆ”‘•–ƒ–‡•‹‹–‹ƒ–‡†„›‡˜‡–•‹–Š‡•–ƒ–‡‘†‡Ž
ˆ‘”™Š‹…Š‘˜ƒŽ‹†–”ƒ•‹–‹‘‹••’‡…‹ϐ‹‡†ȌǢ
— multiple transitions (N-switch coverage), in which test coverage items shall be derived to cover
valid sequences of N+1 transitions in the state model.
NOTE “1-switch” coverage is a popular variant of “N-switch” coverage that requires pairs of transitions to
be exercised.
16 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

5.2.8.3 Derive Test Cases (TD4)

Test cases for state transition testing shall be derived to exercise the test coverage items. The following
steps shall be used during test case derivation:
a) Select test coverage item(s) for inclusion in the current test case;
„Ȍ †‡–‹ˆ›‹’—–˜ƒŽ—‡•–‘‡š‡”…‹•‡–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ•Ȍ–‘„‡…‘˜‡”‡†„›–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•ȋ–Š‡‡š’‡…–‡†
”‡•—Ž–ƒ›„‡†‡ϐ‹‡†‹–‡”•‘ˆ‘—–’—–•ƒ†–Š‡•–ƒ–‡•˜‹•‹–‡†ƒ•†‡•…”‹„‡†‹–Š‡•–ƒ–‡‘†‡ŽȌǢ
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.2.9 Scenario Testing

5.2.9.1 Derive Test Conditions (TD2)

Scenario testing (Desikan and Ramesh 2007) uses a model of the sequences of interactions between
–Š‡–‡•–‹–‡ƒ†‘–Š‡”•›•–‡•ȋ‹–Š‹•…‘–‡š–—•‡”•ƒ”‡‘ˆ–‡…‘•‹†‡”‡†–‘„‡‘–Š‡”•›•–‡•Ȍˆ‘”–Š‡
’—”’‘•‡‘ˆ–‡•–‹‰—•ƒ‰‡ϐŽ‘™•‹˜‘Ž˜‹‰–Š‡–‡•–‹–‡Ǥ‡•–…‘†‹–‹‘••ŠƒŽŽ‡‹–Š‡”„‡‘‡•‡“—‡…‡‘ˆ
interactions (i.e. one scenario) or all sequences of interactions (i.e. all scenarios).
•…‡ƒ”‹‘–‡•–‹‰ǡ–Š‹••–‡’•ŠƒŽŽ‹…Ž—†‡‹†‡–‹ϐ‹…ƒ–‹‘‘ˆǣ
Ȅ –Š‡Dzƒ‹dz•…‡ƒ”‹‘™Š‹…Š‹•–Š‡–›’‹…ƒŽ•‡“—‡…‡‘ˆƒ…–‹‘•–Šƒ–ƒ”‡‡š’‡…–‡†‘ˆ–Š‡–‡•–‹–‡‘”ƒ
ƒ”„‹–”ƒ”›…Š‘‹…‡™Š‡‘–›’‹…ƒŽ•‡“—‡…‡‘ˆƒ…–‹‘•‹•‘™Ǣƒ†
Ȅ DzƒŽ–‡”ƒ–‹˜‡dz•…‡ƒ”‹‘•–Šƒ–”‡’”‡•‡–ƒŽ–‡”ƒ–‹˜‡ȋ‘Ǧƒ‹Ȍ•…‡ƒ”‹‘•–Šƒ–ƒ›„‡–ƒ‡–Š”‘—‰Š
the test item.
NOTE 1 Alternative scenarios can include abnormal use, extreme or stress conditions and exceptions.

ʹ …‡ƒ”‹‘–‡•–‹‰‹•–›’‹…ƒŽŽ›—•‡†ˆ‘”…‘†—…–‹‰Dz‡†Ǧ–‘Ǧ‡†–‡•–‹‰dz†—”‹‰ˆ—…–‹‘ƒŽ–‡•–‹‰ǡ•—…Š
ƒ•†—”‹‰•›•–‡–‡•–‹‰‘”ƒ……‡’–ƒ…‡–‡•–‹‰Ǥ

One common form of scenario testing called use case testing (Bath 2008; Hass 2008) utilises a use case
model of the test item that describes how the test item interacts with one or more actors for the purpose
of testing sequences of interactions (i.e. scenarios) involving the test item.
͵  —•‡ …ƒ•‡ –‡•–‹‰ǡ –Š‡ —•‡ …ƒ•‡ ‘†‡Ž —•– „‡ ‘ˆ ƒ –›’‡ –Šƒ– †‡•…”‹„‡• Š‘™ ˜ƒ”‹‘—• ƒ…–‹‘• ƒ”‡
’‡”ˆ‘”‡†„›–Š‡–‡•–‹–‡ƒ•ƒ”‡•—Ž–‘ˆ˜ƒ”‹‘—•–”‹‰‰‡”•ˆ”‘–Š‡ƒ…–‘”•Ǥƒ…–‘”…ƒ„‡ƒ—•‡”‘”ƒ‘–Š‡”•›•–‡Ǥ

Ͷ ”ƒ•ƒ…–‹‘ϐŽ‘™–‡•–‹‰ȋ‡‹œ‡”ͳͻͻͷȌ‹•‘ˆ–‡…Žƒ••‹ϐ‹‡†ƒ•ƒ–›’‡‘ˆ•…‡ƒ”‹‘–‡•–‹‰Ǥ

5.2.9.2 Derive Test Coverage Items (TD3)

The test coverage items shall be the main and alternative scenarios (i.e. the test coverage items are the
same as the test conditions). Therefore, no further action is required at this step for this technique.

5.2.9.3 Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘”•…‡ƒ”‹‘–‡•–‹‰•ŠƒŽŽ„‡†‡”‹˜‡†„›…‘˜‡”‹‰‡ƒ…Š•…‡ƒ”‹‘ȋ–‡•–…‘˜‡”ƒ‰‡‹–‡Ȍ™‹–Šƒ–
least one test case. The following steps shall be used during test case derivation:
a) Select test coverage item(s) to exercise in the current test case;
„Ȍ †‡–‹ˆ›‹’—–˜ƒŽ—‡•–‘‡š‡”…‹•‡–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ•Ȍ…‘˜‡”‡†„›–Š‡–‡•–…ƒ•‡Ǣ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps a) to c) until the required level of test coverage is achieved.
© ISO/IEC 2015 – All rights reserved 17
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.2.10 Random Testing

5.2.10.1 Derive Test Conditions (TD2)

Random testing (BS 7925-2:1998; Craig and Jaskiel 2002; Kaner 1988) uses a model of the input
†‘ƒ‹‘ˆ–Š‡–‡•–‹–‡–Šƒ–†‡ϐ‹‡•–Š‡•‡–‘ˆƒŽŽ’‘••‹„Ž‡‹’—–˜ƒŽ—‡•Ǥ‹’—–†‹•–”‹„—–‹‘ˆ‘”–Š‡
generation of random input values shall be chosen. The entire input domain shall be the test condition
for random testing.
 ›’‡• ‘ˆ ‹’—– †‹•–”‹„—–‹‘• ‹…Ž—†‡ –Š‡ ‘”ƒŽ †‹•–”‹„—–‹‘ǡ —‹ˆ‘” †‹•–”‹„—–‹‘ ƒ†
‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡Ǥ

5.2.10.2 Derive Test Coverage Items (TD3)

There are no recognised test coverage items for random testing.

5.2.10.3 Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘””ƒ†‘–‡•–‹‰•ŠƒŽŽ„‡…Š‘•‡„›”ƒ†‘Ž›•‡Ž‡…–‹‰‹’—–˜ƒŽ—‡•ˆ”‘–Š‡‹’—–†‘ƒ‹
‘ˆ –Š‡ –‡•– ‹–‡ ȋ‘” ’•‡—†‘Ǧ”ƒ†‘Ž› ‹ˆ —•‹‰ ƒ –‘‘ŽȌ ƒ……‘”†‹‰ –‘ –Š‡ …Š‘•‡ ‹’—– †‹•–”‹„—–‹‘Ǥ Š‡
following steps shall be used during test case derivation:
a) Select an input distribution for the selection of test inputs;
b) Generate random values for the test inputs based on the input distribution chosen in step a);
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps b) and c) until the required testing has been completed.
ͳ Š‡”‡“—‹”‡†–‡•–‹‰…ƒ„‡†‡ϐ‹‡†‹–‡”•‘ˆƒ—„‡”‘ˆ–‡•–•‡š‡…—–‡†ǡƒƒ‘—–‘ˆ–‹‡•’‡–
testing or some other measure of completion.

ʹ –‡’„Ȍ‹•‘”ƒŽŽ›ƒ—–‘ƒ–‡†Ǥ

5.3 Structure-Based Test Design Techniques

5.3.1 Statement Testing

5.3.1.1 Derive Test Conditions (TD2)

‘†‡Ž‘ˆ–Š‡•‘—”…‡…‘†‡‘ˆ–Š‡–‡•–‹–‡™Š‹…Š‹†‡–‹ϐ‹‡••–ƒ–‡‡–•ƒ•‡‹–Š‡”‡š‡…—–ƒ„Ž‡‘”‘Ǧ
‡š‡…—–ƒ„Ž‡ •ŠƒŽŽ „‡ †‡”‹˜‡† ȋ ͹ͻʹͷǦʹǣͳͻͻͺǡ ›‡”• ͳͻ͹ͻȌǤ ƒ…Š ‡š‡…—–ƒ„Ž‡ •–ƒ–‡‡– •ŠƒŽŽ „‡ ƒ
test condition.
 Š‡ ‹†‡–‹ϐ‹…ƒ–‹‘ ‘ˆ ‘Ǧ‡š‡…—–ƒ„Ž‡ •–ƒ–‡‡–• …‘—Ž† „‡ …ƒ””‹‡† ‘—– †—”‹‰ •–‡’ Ͷǡ —•‹‰ ƒ
automated tool.

5.3.1.2 Derive Test Coverage Items (TD3)

Each executable statement shall be a test coverage item (i.e. the test coverage items are the same as the
test conditions). Therefore, no further action is required at this step for this technique.

5.3.1.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;

18 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡
corresponding test inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.3.2 Branch Testing

5.3.2.1 Derive Test Conditions (TD2)

 …‘–”‘Ž ϐŽ‘™ ‘†‡Ž –Šƒ– ‹†‡–‹ϐ‹‡• „”ƒ…Š‡• ‹ –Š‡ …‘–”‘Ž ϐŽ‘™ ‘ˆ –Š‡ –‡•– ‹–‡ •ŠƒŽŽ „‡ †‡”‹˜‡†
ȋ͹ͻʹͷǦʹǣͳͻͻͺǢͶǤ͹ǤͳȌǤƒ…Š„”ƒ…Š‹–Š‡…‘–”‘ŽϐŽ‘™‘†‡Ž•ŠƒŽŽ„‡ƒ–‡•–…‘†‹–‹‘Ǥ
A branch is:
Ȅ ƒ…‘†‹–‹‘ƒŽ–”ƒ•ˆ‡”‘ˆ…‘–”‘Žˆ”‘ƒ›‘†‡‹–Š‡…‘–”‘ŽϐŽ‘™‘†‡Ž–‘ƒ›‘–Š‡”‘†‡Ǣ‘”
Ȅ ƒ‡š’Ž‹…‹–—…‘†‹–‹‘ƒŽ–”ƒ•ˆ‡”‘ˆ…‘–”‘Žˆ”‘ƒ›‘†‡–‘ƒ›‘–Š‡”‘†‡‹–Š‡…‘–”‘ŽϐŽ‘™
model; or
Ȅ ™Š‡ƒ–‡•–‹–‡Šƒ•‘”‡–Šƒ‘‡‡–”›’‘‹–ǡƒ–”ƒ•ˆ‡”‘ˆ…‘–”‘Ž–‘ƒ‡–”›’‘‹–‘ˆ–Š‡–‡•–‹–‡Ǥ
NOTE 1 Complete branch testing covering 100% of all branches requires all arcs (links or edges) in the control
ϐŽ‘™‰”ƒ’Š–‘„‡–‡•–‡†ǡ‹…Ž—†‹‰•‡“—‡–‹ƒŽ•–ƒ–‡‡–•„‡–™‡‡ƒ‡–”›ƒ†‡š‹–’‘‹––Šƒ–…‘–ƒ‹•‘†‡…‹•‹‘•Ǥ

ʹ ”ƒ…Š –‡•–‹‰ …ƒ ”‡“—‹”‡ –‡•–‹‰ ‘ˆ „‘–Š …‘†‹–‹‘ƒŽ ƒ† —…‘†‹–‹‘ƒŽ „”ƒ…Š‡•ǡ ‹…Ž—†‹‰ ‡–”›
and exit points to a test item, depending on the level of test coverage required.

͵ —…–‹‘ƒ†‡–Š‘†…ƒŽŽ•ƒ”‡‘–‹†‡–‹ϐ‹‡†ƒ••‡’ƒ”ƒ–‡–‡•–…‘†‹–‹‘•‹„”ƒ…Š–‡•–‹‰Ǥ

5.3.2.2 Derive Test Coverage Items (TD3)

ƒ…Š„”ƒ…Š‹–Š‡…‘–”‘ŽϐŽ‘™‘†‡Ž•ŠƒŽŽ„‡ƒ–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ‹Ǥ‡Ǥ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡–Š‡
same as the test conditions). Therefore, no further action is required at this step for this technique.

5.3.2.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.
NOTE 1 If there are no branches in the test item then a single test condition, test coverage item and test case is
still required.

ʹ ˆ –Š‡”‡ ƒ”‡ —Ž–‹’Ž‡ ‡–”› ’‘‹–• –‘ –Š‡ –‡•– ‹–‡ǡ •—ˆϐ‹…‹‡– –‡•– …ƒ•‡• ™‹ŽŽ „‡ ”‡“—‹”‡† –‘ …‘˜‡”
‡ƒ…Š‡–”›’‘‹–Ǥ

© ISO/IEC 2015 – All rights reserved 19

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.3.3 Decision Testing

5.3.3.1 Derive Test Conditions (TD2)

…‘–”‘ŽϐŽ‘™‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–‹†‡–‹ϐ‹‡•†‡…‹•‹‘••ŠƒŽŽ„‡†‡”‹˜‡†Ǥ‡…‹•‹‘•ƒ”‡’‘‹–•‹
–Š‡–‡•–‹–‡™Š‡”‡–™‘‘”‘”‡’‘••‹„Ž‡‘—–…‘‡•ȋƒ†Š‡…‡•—„Ǧ’ƒ–Š•Ȍƒ›„‡–ƒ‡„›–Š‡…‘–”‘Ž
ϐŽ‘™ ȋ ͹ͻʹͷǦʹǣͳͻͻͺǡ ›‡”• ͳͻ͹ͻȌǤ ›’‹…ƒŽ †‡…‹•‹‘• ƒ”‡ —•‡† ˆ‘” •‹’Ž‡ •‡Ž‡…–‹‘• ȋ‡Ǥ‰Ǥ ‹ˆǦ–Š‡Ǧ‡Ž•‡
in source code), to decide when to exit loops (e.g. while-loop in source code), and in case (switch)
•–ƒ–‡‡–• ȋ‡Ǥ‰Ǥ …ƒ•‡ǦͳǦʹǦ͵ǦǤǤǤǦ ‹ •‘—”…‡ …‘†‡ȌǤ  †‡…‹•‹‘ –‡•–‹‰ǡ ‡ƒ…Š †‡…‹•‹‘ ‹ –Š‡ …‘–”‘Ž ϐŽ‘™
model shall be a test condition.

5.3.3.2 Derive Test Coverage Items (TD3)

Š‡†‡…‹•‹‘‘—–…‘‡•ˆ”‘‡ƒ…Š†‡…‹•‹‘•ŠƒŽŽ„‡‹†‡–‹ϐ‹‡†ƒ•–‡•–…‘˜‡”ƒ‰‡‹–‡•Ǥ

5.3.3.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.
NOTE If there are no decisions in the test item then a single test condition, test coverage item and test case is
still required.

5.3.4 Branch Condition Testing

5.3.4.1 Derive Test Conditions (TD2)

…‘–”‘ŽϐŽ‘™‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–‹†‡–‹ϐ‹‡•†‡…‹•‹‘•ƒ†…‘†‹–‹‘•™‹–Š‹†‡…‹•‹‘••ŠƒŽŽ„‡
derived. Decisions are points in the test item where two or more possible outcomes (and hence sub-
’ƒ–Š•Ȍƒ›„‡–ƒ‡„›–Š‡…‘–”‘ŽϐŽ‘™ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ›‡”•ͳͻ͹ͻȌǤ›’‹…ƒŽ†‡…‹•‹‘•ƒ”‡—•‡†ˆ‘”
simple selections (e.g. if-then-else in source code), for deciding when to exit loops (e.g. while-loop in
source code), and for case statements (e.g. case-1-2-3-...-N in source code, also referred to as switch
statements). In branch condition testing (BS 7925-2:1998), each decision shall be a test condition.
  ’”‘‰”ƒ •‘—”…‡ …‘†‡ –Š‡ †‡…‹•‹‘ •–ƒ–‡‡– Dz‹ˆ      –Š‡dz ‹• ƒ –‡•– …‘†‹–‹‘ –Šƒ–
…‘–ƒ‹•–Š”‡‡…‘†‹–‹‘•”‡Žƒ–‡†„›Ž‘‰‹…ƒŽ‘’‡”ƒ–‘”•Ǥ

5.3.4.2 Derive Test Coverage Items (TD3)

In branch condition testing, all Boolean values (true/false) of the condition(s) within decisions shall be
‹†‡–‹ϐ‹‡†ƒ•–‡•–…‘˜‡”ƒ‰‡‹–‡•ǤŠ‡†‡…‹•‹‘‘—–…‘‡•ˆ”‘‡ƒ…Š†‡…‹•‹‘•ŠƒŽŽƒŽ•‘„‡‹†‡–‹ϐ‹‡†ƒ•
test coverage items.

5.3.4.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–…‘˜‡”‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
20 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

…Ȍ †‡–‹ˆ›ƒ•—„•‡–‘ˆ–‡•–‹’—–•ˆ”‘•–‡’„Ȍ–‘…‘˜‡”–Š‡‘‘Ž‡ƒ˜ƒŽ—‡•‘ˆ…‘†‹–‹‘•™‹–Š‹–Š‡
decision and the decision outcomes;
†Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
e) Repeat steps a) to d) until the required level of test coverage is achieved.
NOTE If there are no decisions in the test item then a single test condition, test coverage item and test case is
still required.

5.3.5 Branch Condition Combination Testing

5.3.5.1 Derive Test Conditions (TD2)

 …‘–”‘Ž ϐŽ‘™ ‘†‡Ž ‘ˆ –Š‡ –‡•– ‹–‡ ™Š‹…Š ‹†‡–‹ϐ‹‡• †‡…‹•‹‘• ƒ† …‘†‹–‹‘• •ŠƒŽŽ „‡ †‡”‹˜‡†Ǥ 
branch condition combination testing (BS 7925-2:1998), each decision shall be a test condition.
  ’”‘‰”ƒ •‘—”…‡ …‘†‡ –Š‡ †‡…‹•‹‘ •–ƒ–‡‡– Dz‹ˆ      –Š‡dz ‹• ƒ –‡•– …‘†‹–‹‘ –Šƒ–
…‘–ƒ‹•–Š”‡‡…‘†‹–‹‘•”‡Žƒ–‡†„›Ž‘‰‹…ƒŽ‘’‡”ƒ–‘”•Ǥ

5.3.5.2 Derive Test Coverage Items (TD3)

Each unique feasible combination of Boolean values of conditions within each decision shall be
‹†‡–‹ϐ‹‡†ƒ•ƒ–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ›‡”•ͳͻ͹ͻȌǤŠ‹•‹…Ž—†‡••‹’Ž‡†‡…‹•‹‘•ǡ™Š‡”‡
combinations consist of two individual Boolean outcomes of a single condition within a decision.

5.3.5.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;
„Ȍ ‡–‡”‹‡–Š‡•‡–‘ˆ–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡‹†‡–‹ϐ‹‡†…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ †‡–‹ˆ›ƒ•—„•‡–‘ˆ–‡•–‹’—–•ˆ”‘•–‡’„Ȍ–‘…‘˜‡”–Š‡•‡Ž‡…–‡†…‘„‹ƒ–‹‘‘ˆ‘‘Ž‡ƒ˜ƒŽ—‡•‘ˆ
conditions within the decision;
†Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–„›ƒ’’Ž›‹‰–Š‡•‡Ž‡…–‡†–‡•–‹’—–•–‘–Š‡–‡•–„ƒ•‹•Ǣ
e) Repeat steps a) to d) until the required level of test coverage is achieved.
NOTE If there are no decisions in the test item then a single test condition, test coverage item and test case is
still required.

ͷǤ͵Ǥ͸ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡ȋȌ‡•–‹‰

5.3.6.1 Derive Test Conditions (TD2)

…‘–”‘ŽϐŽ‘™‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–‹†‡–‹ϐ‹‡•†‡…‹•‹‘•ƒ†…‘†‹–‹‘••ŠƒŽŽ„‡†‡”‹˜‡†Ǥ‘†‹ϐ‹‡†
…‘†‹–‹‘†‡…‹•‹‘…‘˜‡”ƒ‰‡ȋȌ–‡•–‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺȌǡ‡ƒ…Š†‡…‹•‹‘•ŠƒŽŽ„‡ƒ–‡•–…‘†‹–‹‘Ǥ
  ’”‘‰”ƒ •‘—”…‡ …‘†‡ –Š‡ †‡…‹•‹‘ •–ƒ–‡‡– Dz‹ˆ      –Š‡dz ‹• ƒ –‡•– …‘†‹–‹‘ –Šƒ–
…‘–ƒ‹•–Š”‡‡…‘†‹–‹‘•”‡Žƒ–‡†„›Ž‘‰‹…ƒŽ‘’‡”ƒ–‘”•Ǥ

5.3.6.2 Derive Test Coverage Items (TD3)

Each unique feasible combination of individual Boolean values of conditions within a decision that
ƒŽŽ‘™•ƒ•‹‰Ž‡‘‘Ž‡ƒ…‘†‹–‹‘–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡†‡…‹•‹‘•ŠƒŽŽ„‡‹†‡–‹ϐ‹‡†

© ISO/IEC 2015 – All rights reserved 21

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

ƒ•ƒ–‡•–…‘˜‡”ƒ‰‡‹–‡ǤŠ‡–Š‡”ƒ…‘†‹–‹‘‹†‡’‡†‡–Ž›ƒˆˆ‡…–•–Š‡‘—–…‘‡‘ˆƒ†‡…‹•‹‘‹••Š‘™
„›˜ƒ”›‹‰‘Ž›–Šƒ–…‘†‹–‹‘™Š‹Ž‡Š‘Ž†‹‰ƒŽŽ‘–Š‡”’‘••‹„Ž‡…‘†‹–‹‘•–‘ϐ‹š‡†•–ƒ–‡•Ǥ
NOTE This includes simple decisions, where combinations consist of two individual Boolean outcomes of a
single condition within a decision.

5.3.6.3 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡
executed during testing;
„Ȍ ‡–‡”‹‡–Š‡•‡–‘ˆ–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡‹†‡–‹ϐ‹‡†…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ †‡–‹ˆ›ƒ•—„•‡–‘ˆ–‡•–‹’—–•ˆ”‘•–‡’„Ȍ–‘…‘˜‡”–Š‡•‡Ž‡…–‡†…‘„‹ƒ–‹‘•‘ˆ‹†‹˜‹†—ƒŽ‘‘Ž‡ƒ
˜ƒŽ—‡• ‘ˆ …‘†‹–‹‘• ™‹–Š‹ –Š‡ †‡…‹•‹‘ –Šƒ– ƒŽŽ‘™ ƒ •‹‰Ž‡ ‘‘Ž‡ƒ …‘†‹–‹‘ –‘ ‹†‡’‡†‡–Ž›
affect the outcome;
†Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–„›ƒ’’Ž›‹‰–Š‡•‡Ž‡…–‡†–‡•–‹’—–•–‘–Š‡–‡•–„ƒ•‹•Ǣ
e) Repeat steps a) to d) until the required level of test coverage is achieved.
NOTE If there are no decisions in the test item then a single test condition, test coverage item and test case is
still required.

5.3.7 Data Flow Testing

5.3.7.1 Derive Test Conditions (TD2)

†ƒ–ƒϐŽ‘™–‡•–‹‰ȋ—”•–‡‹ʹͲͲ͵Ȍǡƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡•ŠƒŽŽ„‡†‡”‹˜‡†–Šƒ–‹†‡–‹ϐ‹‡•…‘–”‘Ž
ϐŽ‘™ •—„Ǧ’ƒ–Š• –Š”‘—‰Š –Š‡ –‡•– ‹–‡ǡ ™‹–Š‹ ™Š‹…Š ‡ƒ…Š †‡ϐ‹‹–‹‘ ‘ˆ ƒ ‰‹˜‡ ˜ƒ”‹ƒ„Ž‡ ‹• Ž‹‡† –‘
•—„•‡“—‡–—•‡ȋ•Ȍ‘ˆ–Š‡•ƒ‡˜ƒ”‹ƒ„Ž‡ƒ†™‹–Š‹™Š‹…Š–Š‡”‡Šƒ•„‡‡‘‹–‡”˜‡‹‰”‡†‡ϐ‹‹–‹‘‘ˆ
the variable’s value.
Dz‡ϐ‹‹–‹‘•dzƒ”‡™Š‡”‡ƒ˜ƒ”‹ƒ„Ž‡‹•’‘••‹„Ž›‰‹˜‡ƒ‡™˜ƒŽ—‡ȋ•‘‡–‹‡•ƒ†‡ϐ‹‹–‹‘™‹ŽŽ”‡•—Ž–‹ƒ
variable retaining the same value as it had before). A “use” is an occurrence of a variable in which the
variable is not given a new value; “uses” can be further distinguished as either “p-uses” (predicate-use)
or “c-uses” (computation-use). A p-use denotes the use of a variable in determining the outcome of a
condition (predicate) within a decision, such as a while-loop, if- then- else, etc. A c-use occurs when a
˜ƒ”‹ƒ„Ž‡‹•—•‡†ƒ•ƒ‹’—––‘–Š‡…‘’—–ƒ–‹‘‘ˆ–Š‡†‡ϐ‹‹–‹‘‘ˆƒ›˜ƒ”‹ƒ„Ž‡‘”‘ˆƒ‘—–’—–Ǥ
†ƒ–ƒϐŽ‘™–‡•–‹‰ǡ‡ƒ…Š†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”ˆ‘”ƒ˜ƒ”‹ƒ„Ž‡‹–Š‡–‡•–‹–‡‹•ƒ–‡•–…‘†‹–‹‘Ǥ
Š‡”‡ƒ”‡ƒ—„‡”‘ˆˆ‘”•‘ˆ†ƒ–ƒϐŽ‘™–‡•–‹‰ǡ™Š‹…Šƒ”‡ƒŽŽ„ƒ•‡†‘–Š‡•ƒ‡–‡•–…‘†‹–‹‘•ǤŠ‡
ϐ‹˜‡ˆ‘”•†‡ϐ‹‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻƒ”‡ǣƒŽŽǦ†‡ϐ‹‹–‹‘•–‡•–‹‰ǡƒŽŽǦ…Ǧ—•‡•–‡•–‹‰ǡƒŽŽǦ
p-uses testing, all-uses testing, and all-du-paths testing.

ͷǤ͵Ǥ͹Ǥʹ ŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

5.3.7.2.1 Derive Test Coverage Items (TD3)

Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ‡ƒ…Š ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ •‘‡ —•‡ ȋ‡‹–Š‡” ’Ǧ—•‡ ‘” …Ǧ—•‡Ȍ ‘ˆ –Šƒ–
†‡ϐ‹‹–‹‘•ŠƒŽŽ„‡‹†‡–‹ϐ‹‡†ƒ•–‡•–…‘˜‡”ƒ‰‡‹–‡•Ǥƒ…Š•—„Ǧ’ƒ–Š‹•‘™ƒ•ƒDz†‡ϐ‹‹–‹‘Ǧ—•‡dz’ƒ–ŠǤDzŽŽǦ
†‡ϐ‹‹–‹‘•dz–‡•–‹‰”‡“—‹”‡•–Šƒ–ƒ–Ž‡ƒ•–‘‡†‡ϐ‹‹–‹‘Ǧˆ”‡‡•—„Ǧ’ƒ–Šȋ™‹–Š”‡Žƒ–‹‘–‘ƒ•’‡…‹ϐ‹…˜ƒ”‹ƒ„Ž‡Ȍ
ˆ”‘–Š‡†‡ϐ‹‹–‹‘–‘‘‡‘ˆ‹–•…Ǧ—•‡•‘”’Ǧ—•‡•™‹ŽŽŠƒ˜‡„‡‡…‘˜‡”‡†ˆ‘”ƒŽŽ˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•Ǥ

22 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

5.3.7.2.2 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›†‡ϐ‹‹–‹‘•–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰Ǣ
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍˆ”‘–Š‡‹†‡–‹ϐ‹‡††‡ϐ‹‹–‹‘•
to be exercised;
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.
NOTE In practice, these steps could require automation.

5.3.7.3 All-C-Uses Testing

5.3.7.3.1 Derive Test Coverage Items (TD3)

Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ‡ƒ…Š ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ ‡ƒ…Š …Ǧ—•‡ ‘ˆ –Šƒ– †‡ϐ‹‹–‹‘ •ŠƒŽŽ „‡
‹†‡–‹ϐ‹‡† ƒ• –‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ǥ DzŽŽǦ…Ǧ—•‡•dz –‡•–‹‰ ”‡“—‹”‡• –Šƒ– ƒ– Ž‡ƒ•– ‘‡ ‘ˆ –Š‡ †‡ϐ‹‹–‹‘Ǧˆ”‡‡
•—„Ǧ’ƒ–Š•ȋ™‹–Š”‡Žƒ–‹‘–‘ƒ•’‡…‹ϐ‹…˜ƒ”‹ƒ„Ž‡Ȍˆ”‘–Š‡†‡ϐ‹‹–‹‘–‘‡ƒ…Š‘‡‘ˆ‹–•…Ǧ—•‡•™‹ŽŽŠƒ˜‡
„‡‡…‘˜‡”‡†ˆ‘”ƒŽŽ”‡Ž‡˜ƒ–˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•Ǥ

5.3.7.3.2 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍˆ”‘˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•–‘ƒ•—„•‡“—‡–…Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘
ȋ™‹–Š‘‹–‡”˜‡‹‰†‡ϐ‹‹–‹‘•Ȍ–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰Ǣ
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.3.7.4 All-P-Uses Testing

5.3.7.4.1 Derive Test Coverage Items (TD3)

Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ‡ƒ…Š ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ ‡ƒ…Š ’Ǧ—•‡ ‘ˆ –Šƒ– †‡ϐ‹‹–‹‘ •ŠƒŽŽ „‡
‹†‡–‹ϐ‹‡† ƒ• –‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ǥ DzŽŽǦ’Ǧ—•‡•dz –‡•–‹‰ ”‡“—‹”‡• –Šƒ– ƒ– Ž‡ƒ•– ‘‡ ‘ˆ –Š‡ †‡ϐ‹‹–‹‘Ǧˆ”‡‡
•—„Ǧ’ƒ–Š•ȋ™‹–Š”‡Žƒ–‹‘–‘ƒ•’‡…‹ϐ‹…˜ƒ”‹ƒ„Ž‡Ȍˆ”‘–Š‡†‡ϐ‹‹–‹‘–‘‡ƒ…Š‘‡‘ˆ‹–•’Ǧ—•‡•™‹ŽŽŠƒ˜‡
„‡‡…‘˜‡”‡†ˆ‘”ƒŽŽ”‡Ž‡˜ƒ–˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•Ǥ

5.3.7.4.2 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍˆ”‘˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•–‘•—„•‡“—‡–’Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘
ȋ™‹–Š‘‹–‡”˜‡‹‰†‡ϐ‹‹–‹‘•Ȍ–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰Ǣ
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.
© ISO/IEC 2015 – All rights reserved 23
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

5.3.7.5 All-Uses Testing

5.3.7.5.1 Derive Test Coverage Items (TD3)

Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ‡ƒ…Š ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ ‡˜‡”› —•‡ ȋ„‘–Š ’Ǧ—•‡ ƒ† …Ǧ—•‡Ȍ ‘ˆ –Šƒ–
†‡ϐ‹‹–‹‘ •ŠƒŽŽ „‡ ‹†‡–‹ϐ‹‡† ƒ• –‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ǥ DzŽŽǦ—•‡•dz –‡•–‹‰ ”‡“—‹”‡• –Šƒ– ƒ– Ž‡ƒ•– ‘‡ •—„Ǧ
’ƒ–Šˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡ƒ…Š‘‡‘ˆ‹–•—•‡•–Šƒ–…ƒ„‡”‡ƒ…Š‡†ȋ™‹–Š‘—–ƒ‹–‡”˜‡‹‰
†‡ϐ‹‹–‹‘‘ˆ–Š‡˜ƒ”‹ƒ„Ž‡Ȍ‹•…‘˜‡”‡†Ǥ

5.3.7.5.2 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍˆ”‘˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•–‘ƒ•—„•‡“—‡–’Ǧ—•‡‘”…Ǧ—•‡‘ˆ–Šƒ–
†‡ϐ‹‹–‹‘–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰Ǣ
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.

5.3.7.6 All-DU-Paths Testing

5.3.7.6.1 Derive Test Coverage Items (TD3)

Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ‡ƒ…Š ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ ‡˜‡”› —•‡ ȋ„‘–Š ’Ǧ—•‡ ƒ† …Ǧ—•‡Ȍ ‘ˆ –Šƒ–
†‡ϐ‹‹–‹‘•ŠƒŽŽ„‡‹†‡–‹ϐ‹‡†ƒ•–‡•–…‘˜‡”ƒ‰‡‹–‡•ǤDzŽŽǦ†—Ǧ’ƒ–Š•dz–‡•–‹‰”‡“—‹”‡•–Šƒ–ƒŽŽ•—„Ǧ’ƒ–Š•
ˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡ƒ…Š‘ˆ‹–•—•‡•–Šƒ–…ƒ„‡”‡ƒ…Š‡†ȋ™‹–Š‘—–ƒ‹–‡”˜‡‹‰†‡ϐ‹‹–‹‘
for the variable) is covered.
NOTE All-du-paths testing requires all Ž‘‘’Ǧˆ”‡‡ •—„Ǧ’ƒ–Š• ˆ”‘ ƒ ˜ƒ”‹ƒ„Ž‡ †‡ϐ‹‹–‹‘ –‘ ‹–• —•‡ „‡ –‡•–‡†
–‘ƒ––‡’––‘ƒ…Š‹‡˜‡ͳͲͲΨ–‡•–‹–‡…‘˜‡”ƒ‰‡ǤŠ‹•†‹ˆˆ‡”•ˆ”‘ŽŽǦ—•‡•–‡•–‹‰ǡ™Š‹…Š”‡“—‹”‡•‘Ž›one path
ˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‹–•—•‡–‘„‡–‡•–‡†–‘ƒ––‡’––‘ƒ…Š‹‡˜‡ͳͲͲΨ–‡•–‹–‡…‘˜‡”ƒ‰‡Ǥ

5.3.7.6.2 Derive Test Cases (TD4)

The following steps shall be followed during test case derivation:

ƒȌ †‡–‹ˆ›…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍˆ”‘˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•–‘‡˜‡”›•—„•‡“—‡–’Ǧ—•‡ƒ†…Ǧ—•‡‘ˆ
–Šƒ–†‡ϐ‹‹–‹‘–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰Ǣ
„Ȍ ‡–‡”‹‡–Š‡–‡•–‹’—–•–Šƒ–™‹ŽŽ…ƒ—•‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šȋ•Ȍ–‘„‡‡š‡”…‹•‡†Ǣ
…Ȍ ‡–‡”‹‡ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ˆ”‘ ‡š‡”…‹•‹‰ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Šȋ•Ȍ „› ƒ’’Ž›‹‰ –Š‡ –‡•–
inputs to the test basis;
d) Repeat steps a) to c) until the required level of test coverage is achieved.

24 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

5.4 Experience-Based Test Design Techniques

5.4.1 Error Guessing

5.4.1.1 Derive Test Conditions (TD2)

””‘”‰—‡••‹‰ȋ͹ͻʹͷǦʹǣͳͻͻͺǡ›‡”•ͳͻ͹ͻȌ‹˜‘Ž˜‡•–Š‡†‡•‹‰‘ˆƒ…Š‡…Ž‹•–‘ˆ†‡ˆ‡…––›’‡•–Šƒ–ƒ›
‡š‹•–‹–Š‡–‡•–‹–‡ǡƒŽŽ‘™‹‰–Š‡–‡•–‡”–‘‹†‡–‹ˆ›‹’—–•–‘–Š‡–‡•–‹–‡–Šƒ–ƒ›…ƒ—•‡ˆƒ‹Ž—”‡•ǡ‹ˆ
–Š‘•‡†‡ˆ‡…–•‡š‹•–‹–Š‡–‡•–‹–‡Ǥƒ…Š†‡ˆ‡…––›’‡•ŠƒŽŽ„‡ƒ–‡•–…‘†‹–‹‘Ǥ
 Š‡ …Š‡…Ž‹•– ‘ˆ †‡ˆ‡…– –›’‡• …‘—Ž† „‡ †‡”‹˜‡† „› ˜ƒ”‹‘—• ‡ƒ•ǡ •—…Š ƒ• –ƒš‘‘‹‡• ‘ˆ ‘™
‡””‘”•ǡ ‹ˆ‘”ƒ–‹‘ …‘–ƒ‹‡† ‹ ‹…‹†‡– ƒƒ‰‡‡– •›•–‡•ǡ ˆ”‘ ƒ –‡•–‡”ǯ• ‘™Ž‡†‰‡ǡ ‡š’‡”‹‡…‡ ƒ†Ȁ‘”
understanding of the test item(s) and/or similar test items or from the knowledge of other stakeholders (e.g.
•›•–‡—•‡”•‘”’”‘‰”ƒ‡”•ȌǤ

5.4.1.2 Derive Test Coverage Items (TD3)

There are no recognised test coverage items for error guessing.

5.4.1.3 Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘”‡””‘”‰—‡••‹‰ƒ”‡–›’‹…ƒŽŽ›†‡”‹˜‡†„›•‡Ž‡…–‹‰ƒ†‡ˆ‡…––›’‡ˆ”‘ƒ…Š‡…Ž‹•–‘ˆ†‡ˆ‡…–
–›’‡•–Šƒ–ƒ”‡–‘„‡…‘˜‡”‡†ƒ††‡”‹˜‹‰–‡•–…ƒ•‡•–Šƒ–…‘—Ž††‡–‡…––Šƒ–†‡ˆ‡…––›’‡‹–Š‡–‡•–‹–‡ǡ‹ˆ
it existed. The following steps shall be used during test case derivation:
ƒȌ ‡Ž‡…–ƒ†‡ˆ‡…––›’‡ȋ•Ȍˆ‘”…‘˜‡”ƒ‰‡„›–Š‡…—””‡––‡•–…ƒ•‡Ǣ
„Ȍ †‡–‹ˆ› ‹’—– ˜ƒŽ—‡• –Šƒ– …‘—Ž† „‡ ‡š’‡…–‡† –‘ …ƒ—•‡ ƒ ˆƒ‹Ž—”‡ …‘””‡•’‘†‹‰ –‘ –Š‡ •‡Ž‡…–‡†
†‡ˆ‡…––›’‡ȋ•ȌǢ
…Ȍ ‡–‡”‹‡–Š‡‡š’‡…–‡†”‡•—Ž–‘ˆ–Š‡–‡•–…ƒ•‡„›ƒ’’Ž›‹‰–Š‡‹’—–ȋ•Ȍ–‘–Š‡–‡•–„ƒ•‹•Ǣ
d) Repeat steps a) to c) until the required testing has been completed.

6 Test Coverage Measurement

6.1 Overview
Š‡…‘˜‡”ƒ‰‡‡ƒ•—”‡•†‡ϐ‹‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻƒ”‡„ƒ•‡†‘†‹ˆˆ‡”‹‰†‡‰”‡‡•‘ˆ
…‘˜‡”ƒ‰‡–Šƒ–ƒ”‡ƒ…Š‹‡˜ƒ„Ž‡„›–‡•–†‡•‹‰–‡…Š‹“—‡•Ǥ‘˜‡”ƒ‰‡Ž‡˜‡Ž•…ƒ”ƒ‰‡ˆ”‘ͲΨ–‘ͳͲͲΨǤ
‡ƒ…Š…‘˜‡”ƒ‰‡…ƒŽ…—Žƒ–‹‘ǡƒ—„‡”‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ›„‡‹ˆ‡ƒ•‹„Ž‡Ǥ–‡•–…‘˜‡”ƒ‰‡‹–‡•ŠƒŽŽ
„‡†‡ϐ‹‡†–‘„‡‹ˆ‡ƒ•‹„Ž‡‹ˆ‹–…ƒ„‡•Š‘™–‘‘–„‡‡š‡…—–ƒ„Ž‡‘”‹’‘••‹„Ž‡–‘„‡…‘˜‡”‡†„›ƒ–‡•–
…ƒ•‡ǤŠ‡…‘˜‡”ƒ‰‡…ƒŽ…—Žƒ–‹‘•ŠƒŽŽ„‡†‡ϐ‹‡†ƒ•‡‹–Š‡”…‘—–‹‰‘”†‹•…‘—–‹‰‹ˆ‡ƒ•‹„Ž‡‹–‡•Ǣ–Š‹•
…Š‘‹…‡™‹ŽŽ–›’‹…ƒŽŽ›„‡”‡…‘”†‡†‹–Š‡‡•–Žƒȋ†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦ͵ȌǤˆƒ–‡•–…‘˜‡”ƒ‰‡
‹–‡‹•†‹•…‘—–‡†ǡŒ—•–‹ϐ‹…ƒ–‹‘ˆ‘”‹–•‹ˆ‡ƒ•‹„‹Ž‹–›™‹ŽŽ–›’‹…ƒŽŽ›„‡”‡…‘”†‡†‹ƒ–‡•–”‡’‘”–Ǥ‡ƒ…Š
…‘˜‡”ƒ‰‡…ƒŽ…—Žƒ–‹‘ǡ‹ˆ–Š‡”‡ƒ”‡‘–‡•–…‘˜‡”ƒ‰‡‹–‡•‘ˆƒ‰‹˜‡–›’‡‹ƒ–‡•–‹–‡ǡͳͲͲΨ…‘˜‡”ƒ‰‡ˆ‘”
–Šƒ––›’‡‘ˆ…‘˜‡”ƒ‰‡™‹ŽŽ„‡†‡ϐ‹‡†ƒ•„‡‹‰‹ƒ’’Ž‹…ƒ„Ž‡ˆ‘”–Šƒ––‡•–‹–‡Ǥ
Š‡…ƒŽ…—Žƒ–‹‰…‘˜‡”ƒ‰‡ˆ‘”ƒ›–‡…Š‹“—‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰ˆ‘”—Žƒ•ŠƒŽŽ„‡—•‡†ǣ

⎛N ⎞
Coverage = ⎜ × 100 ⎟ %
⎝ T ⎠
where:
Ȅ ‘˜‡”ƒ‰‡‹•–Š‡…‘˜‡”ƒ‰‡ƒ…Š‹‡˜‡†„›ƒ•’‡…‹ϐ‹…–‡•–†‡•‹‰–‡…Š‹“—‡Ǣ
Ȅ ‹•–Š‡—„‡”‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ

© ISO/IEC 2015 – All rights reserved 25

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•‹†‡–‹ϐ‹‡†„›–Š‡–‡•–†‡•‹‰–‡…Š‹“—‡Ǥ
’‡…‹ϐ‹… ˜ƒŽ—‡• ˆ‘” ‘˜‡”ƒ‰‡ǡ  ƒ†  ˆ‘” ‡ƒ…Š –‡…Š‹“—‡ ƒ”‡ †‡ϐ‹‡† ‹ –Š‡ …Žƒ—•‡• „‡Ž‘™Ǥ Š‡ •‡– ‘ˆ
coverage measures presented in the following clauses are designed to be used with the test design
–‡…Š‹“—‡• ’”‡•‡–‡† ‹ –Š‹• •–ƒ†ƒ”†Ǥ Š‡› ƒ”‡ ‘– †‡•‹‰‡† –‘ „‡ ƒ ‡šŠƒ—•–‹˜‡ Ž‹•–Ǣ –Š‡”‡ ƒ› „‡
‘–Š‡”…‘˜‡”ƒ‰‡‡ƒ•—”‡•–Šƒ–ƒ”‡—•‡†„›‘”‰ƒ‹œƒ–‹‘•–Šƒ–ƒ”‡‘–‡–‹‘‡†‹–Š‹•…Žƒ—•‡Ǥ
 –Š‡” ‡ƒ•—”‡• –Šƒ– …‘—Ž† „‡ ”‡“—‹”‡† ˆ‘” ƒ••‡••‹‰ –Š‡ …‘’Ž‡–‡‡•• ‘ˆ –‡•–‹‰ ‹…Ž—†‡
measuring the overall percentage of requirements that have been covered during testing.

͸Ǥʹ ‡•–‡ƒ•—”‡‡–ˆ‘”’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡•

6.2.1 Equivalence Partition Coverage

‘˜‡”ƒ‰‡ˆ‘”‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is the equivalence partition coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ’ƒ”–‹–‹‘•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ’ƒ”–‹–‹‘•‹†‡–‹ϐ‹‡†Ǥ

͸ǤʹǤʹ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†‘˜‡”ƒ‰‡

‘˜‡”ƒ‰‡ˆ‘”–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•Ǥ
‘”‹‹‹œ‡†…‘„‹ƒ–‹‘…‘˜‡”ƒ‰‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘••ŠƒŽŽ„‡—•‡†ǣ
Ȅ ‘˜‡”ƒ‰‡‹•–Š‡‹‹‹œ‡†…‘„‹ƒ–‹‘…‘˜‡”ƒ‰‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†…‘˜‡”ƒ‰‡Ǣ
Ȅ ‹•–Š‡—„‡”‘ˆ…Žƒ••‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of classes.
‘”ƒš‹‹œ‡†…‘„‹ƒ–‹‘…‘˜‡”ƒ‰‡–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘••ŠƒŽŽ„‡—•‡†ǣ
Ȅ ‘˜‡”ƒ‰‡‹•–Š‡ƒš‹‹œ‡†…‘„‹ƒ–‹‘…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†…‘˜‡”ƒ‰‡Ǣ
Ȅ ‹•–Š‡—„‡”‘ˆ…‘„‹ƒ–‹‘•‘ˆ…Žƒ••‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of combinations of classes.

6.2.3 Boundary Value Analysis Coverage

‘˜‡”ƒ‰‡ˆ‘”„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹••ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
Ȅ ‘˜‡”ƒ‰‡‹•–Š‡„‘—†ƒ”›˜ƒŽ—‡…‘˜‡”ƒ‰‡Ǣ
Ȅ ‹•–Š‡—„‡”‘ˆ†‹•–‹…–„‘—†ƒ”›˜ƒŽ—‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ„‘—†ƒ”›˜ƒŽ—‡•Ǥ
Š‡†‡…‹•‹‘–‘ƒ’’Ž›–™‘Ǧ˜ƒŽ—‡‘”–Š”‡‡Ǧ˜ƒŽ—‡„‘—†ƒ”›–‡•–‹‰•Š‘—Ž†„‡”‡…‘”†‡†Ǥ

6.2.4 Syntax Testing Coverage

Š‡”‡‹•…—””‡–Ž›‘‹†—•–”›ƒ‰”‡‡†ƒ’’”‘ƒ…Š–‘…ƒŽ…—Žƒ–‹‰…‘˜‡”ƒ‰‡ˆ‘”•›–ƒš–‡•–‹‰Ǥ
 ƒŽ…—Žƒ–‹‰…‘˜‡”ƒ‰‡ƒ•ƒ’‡”…‡–ƒ‰‡‹•‘–’‘••‹„Ž‡ˆ‘”•›–ƒš–‡•–‹‰†—‡–‘–Š‡’‘–‡–‹ƒŽŽ›‡š–”‡‡Ž›
Žƒ”‰‡—„‡”‘ˆ‘’–‹‘•ƒ†‹ϐ‹‹–‡—„‡”‘ˆ—–ƒ–‹‘•’‘••‹„Ž‡Ǥ

26 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

6.2.5 Combinatorial Test Design Technique Coverage

6.2.5.1 All Combinations Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽ…‘„‹ƒ–‹‘•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all combinations coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ—‹“—‡…‘„‹ƒ–‹‘•‘ˆǦ’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of unique P-V pair combinations (the product of the number of P-V pairs for
each test item parameter).
 ‘”ƒ†‡ϐ‹‹–‹‘‘ˆǦ’ƒ‹”ǡ”‡ˆ‡”–‘ͶǤʹǤ͵Ǥ

6.2.5.2 Pair-wise Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”’ƒ‹”Ǧ™‹•‡–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is pair-wise coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ—‹“—‡’ƒ‹”•‘ˆǦ’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of unique pairs of P-V pairs.

6.2.5.3 Each Choice Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”‡ƒ…Š…Š‘‹…‡–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is each choice coverage;
Ȅ ‹•–Š‡—„‡”‘ˆǦ’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of unique P-V pairs.

6.2.5.4 Base Choice Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”„ƒ•‡…Š‘‹…‡–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is base choice coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ„ƒ•‡…Š‘‹…‡…‘„‹ƒ–‹‘•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•ȋƒŽŽ„—–‘‡–‡•–‹–‡
parameter set to the base value, and the remaining test item parameter set to valid values), plus one
(for when all test item parameters are set to the base value) if exercised;
— T is the total number of base choice combinations (all but one test item parameter set to the base
value, and the remaining test item parameter set to valid values), plus one (for when all test item
parameters are set to the base value).

6.2.6 Decision Table Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”†‡…‹•‹‘–ƒ„Ž‡–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is decision table coverage;
Ȅ ‹•–Š‡—„‡”‘ˆˆ‡ƒ•‹„Ž‡†‡…‹•‹‘”—Ž‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of feasible decision rules.

© ISO/IEC 2015 – All rights reserved 27

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

6.2.7 Cause-Effect Graphing Coverage

‘˜‡”ƒ‰‡ˆ‘”…ƒ—•‡Ǧ‡ˆˆ‡…–‰”ƒ’Š‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is the cause/effect coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ†‡…‹•‹‘”—Ž‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of feasible decision rules.

6.2.8 State Transition Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽ•–ƒ–‡•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all states coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ•–ƒ–‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of states.
‘˜‡”ƒ‰‡ˆ‘”•‹‰Ž‡–”ƒ•‹–‹‘•ȋͲǦ•™‹–…Š…‘˜‡”ƒ‰‡Ȍ•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is 0-switch coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ•‹‰Ž‡˜ƒŽ‹†–”ƒ•‹–‹‘•‡š‡…—–‡†„›–‡•–…ƒ•‡•Ǣ
— T is the total number of single valid transitions.
‘˜‡”ƒ‰‡ˆ‘”ƒŽŽ–”ƒ•‹–‹‘••ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all transitions transition coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ˜ƒŽ‹†ƒ†‹˜ƒŽ‹†–”ƒ•‹–‹‘•ƒ––‡’–‡†–‘„‡…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ˜ƒŽ‹†ƒ†‹˜ƒŽ‹†–”ƒ•‹–‹‘•„‡–™‡‡‹†‡–‹ϐ‹‡†•–ƒ–‡•‹‹–‹ƒ–‡†„›˜ƒŽ‹†‡˜‡–•Ǥ
‘˜‡”ƒ‰‡ˆ‘”Ϊͳ–”ƒ•‹–‹‘•ȋ•™‹–…Š–‡•–‹‰Ȍ•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is N-switch coverage;
Ȅ ‹•–Š‡—„‡”‘ˆΪͳ˜ƒŽ‹†–”ƒ•‹–‹‘•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of sequences of N+1 valid transitions.

6.2.9 Scenario Testing Coverage

Coverage for scenario testing (including use case testing) shall be calculated using the following
†‡ϐ‹‹–‹‘•ǣ
— Coverage is scenario coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ•…‡ƒ”‹‘•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of scenarios.

6.2.10 Random Testing Coverage

Š‡”‡‹•…—””‡–Ž›‘‹†—•–”›ƒ‰”‡‡†ƒ’’”‘ƒ…Š–‘…ƒŽ…—Žƒ–‹‰…‘˜‡”ƒ‰‡ˆ‘””ƒ†‘–‡•–‹‰Ǥ

28 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

6.3 Test Measurement for Structure-Based Test Design Techniques

6.3.1 Statement Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”•–ƒ–‡‡––‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is statement coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ‡š‡…—–ƒ„Ž‡•–ƒ–‡‡–•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of executable statements.

6.3.2 Branch Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”„”ƒ…Š–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is branch coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ„”ƒ…Š‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of branches.
NOTE For situations where there are no branches in the test item, a single test is required to achieve 100%
branch coverage.

6.3.3 Decision Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”†‡…‹•‹‘–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is decision coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ†‡…‹•‹‘‘—–…‘‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of decision outcomes.
NOTE For situations where there are no decisions in the test item, a single test is required to achieve 100%
decision coverage.

6.3.4 Branch Condition Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”„”ƒ…Š…‘†‹–‹‘–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is branch condition coverage;
— N is the number of Boolean values of conditions within decisions plus the number of decision
‘—–…‘‡•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of Boolean values of conditions within decisions plus the total number of
decision outcomes.
NOTE For situations where there are no decisions in the test item, a single test is required to achieve 100%
branch condition coverage.

6.3.5 Branch Condition Combination Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”„”ƒ…Š…‘†‹–‹‘…‘„‹ƒ–‹‘–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is branch condition combination coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ…‘„‹ƒ–‹‘•‘ˆ‘‘Ž‡ƒ˜ƒŽ—‡•‘ˆ…‘†‹–‹‘•™‹–Š‹‡ƒ…Š†‡…‹•‹‘…‘˜‡”‡†„›
executed test cases;

© ISO/IEC 2015 – All rights reserved 29

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

— T is the total number of unique combinations of Boolean values of conditions within decisions.
NOTE For situations where there are no decisions in the test item, a single test is required to achieve 100%
branch condition combination coverage.

͸Ǥ͵Ǥ͸ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘ȋȌ‡•–‹‰‘˜‡”ƒ‰‡

‘˜‡”ƒ‰‡ ˆ‘” ‘†‹ϐ‹‡† …‘†‹–‹‘ †‡…‹•‹‘ …‘˜‡”ƒ‰‡ –‡•–‹‰ •ŠƒŽŽ „‡ …ƒŽ…—Žƒ–‡† —•‹‰ –Š‡ ˆ‘ŽŽ‘™‹‰
†‡ϐ‹‹–‹‘•ǣ
Ȅ ‘˜‡”ƒ‰‡‹•‘†‹ϐ‹‡†…‘†‹–‹‘†‡…‹•‹‘…‘˜‡”ƒ‰‡Ǣ
— N is the number of unique feasible combinations of individual Boolean values of conditions within
†‡…‹•‹‘•–Šƒ–ƒŽŽ‘™ƒ•‹‰Ž‡‘‘Ž‡ƒ…‘†‹–‹‘–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡†‡…‹•‹‘‘—–…‘‡–‘„‡
…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
— T is the total number of unique combinations of individual Boolean values of conditions within
†‡…‹•‹‘•–Šƒ–ƒŽŽ‘™ƒ•‹‰Ž‡‘‘Ž‡ƒ…‘†‹–‹‘–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡Ǥ
NOTE For situations where there are no decisions in the test item, a single test is required to achieve 100%
‘†‹ϐ‹‡†…‘†‹–‹‘†‡…‹•‹‘…‘˜‡”ƒ‰‡Ǥ

6.3.7 Data Flow Testing Coverage

͸Ǥ͵Ǥ͹Ǥͳ ŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰‘˜‡”ƒ‰‡

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽǦ†‡ϐ‹‹–‹‘•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
Ȅ ‘˜‡”ƒ‰‡‹•ƒŽŽǦ†‡ϐ‹‹–‹‘•…‘˜‡”ƒ‰‡Ǣ
Ȅ ‹•–Š‡—„‡”‘ˆ†‡ϐ‹‹–‹‘•ƒ••‘…‹ƒ–‡†™‹–Š†ƒ–ƒǦ†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–
cases;
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ†‡ϐ‹‹–‹‘•Ǧ—•‡’ƒ‹”•ˆ”‘†‹•–‹…–˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘•Ǥ

6.3.7.2 All-C-Uses Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽǦ…Ǧ—•‡•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all-c-uses coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ†‡ϐ‹‹–‹‘Ǧ…Ǧ—•‡’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ†‡ϐ‹‹–‹‘Ǧ…Ǧ—•‡’ƒ‹”•Ǥ

6.3.7.3 All-P-Uses Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽǦ’Ǧ—•‡•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all-p-uses coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ†‡ϐ‹‹–‹‘Ǧ’Ǧ—•‡’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ†‡ϐ‹‹–‹‘Ǧ’Ǧ—•‡’ƒ‹”•Ǥ

6.3.7.4 All-Uses Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽǦ—•‡•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all-uses coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ—‹“—‡†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
30 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•ˆ”‘‡ƒ…Š†‡ϐ‹‹–‹‘–‘„‘–Š’Ǧ—•‡and c-use of that

†‡ϐ‹‹–‹‘Ǥ

6.3.7.5 All-DU-Paths Testing Coverage

‘˜‡”ƒ‰‡ˆ‘”ƒŽŽǦ†—Ǧ’ƒ–Š•–‡•–‹‰•ŠƒŽŽ„‡…ƒŽ…—Žƒ–‡†—•‹‰–Š‡ˆ‘ŽŽ‘™‹‰†‡ϐ‹‹–‹‘•ǣ
— Coverage is all-du-paths coverage;
Ȅ ‹•–Š‡—„‡”‘ˆ•—„Ǧ’ƒ–Š•ˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡ƒ…Š‘ˆ‹–•—•‡•–Šƒ–…ƒ„‡”‡ƒ…Š‡†
ȋ™‹–Š‘—–ƒ‹–‡”˜‡‹‰†‡ϐ‹‹–‹‘‘ˆ–Š‡˜ƒ”‹ƒ„Ž‡Ȍ…‘˜‡”‡†„›‡š‡…—–‡†–‡•–…ƒ•‡•Ǣ
Ȅ ‹•–Š‡–‘–ƒŽ—„‡”‘ˆ•‹’Ž‡•—„Ǧ’ƒ–Š•ˆ”‘‡ƒ…Š†‡ϐ‹‹–‹‘–‘„‘–Š’Ǧ—•‡and…Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘Ǥ

6.4 Test Measurement for Experience-Based Testing Design Techniques

6.4.1 Error Guessing Coverage

Š‡”‡‹•…—””‡–Ž›‘‹†—•–”›ƒ‰”‡‡†ƒ’’”‘ƒ…Š–‘…ƒŽ…—Žƒ–‹‰…‘˜‡”ƒ‰‡ˆ‘”‡””‘”‰—‡••‹‰Ǥ

© ISO/IEC 2015 – All rights reserved 31

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Annex A
(informative)

Testing Quality Characteristics

A.1 Quality Characteristics

A.1.1 Overview
‘ˆ–™ƒ”‡ –‡•–‹‰ ‹• …ƒ””‹‡† ‘—– –‘ …‘ŽŽ‡…– ‡˜‹†‡…‡ ƒ• –‘ ™Š‡–Š‡” ”‡“—‹”‡† “—ƒŽ‹–› …”‹–‡”‹ƒ Šƒ˜‡ „‡‡
•ƒ–‹•ϐ‹‡† „› ƒ –‡•– ‹–‡Ǥ Š‹• ‹ˆ‘”ƒ–‹˜‡ ƒ‡š …‘–ƒ‹• ‡šƒ’Ž‡• ‘ˆ Š‘™ “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹…•
†‡ϐ‹‡†‹ȀʹͷͲͳͲȋȀʹͷͲͳͲǣʹͲͳͳȌ…ƒ„‡ƒ’’‡†–‘–Š‡–‡•–†‡•‹‰–‡…Š‹“—‡•†‡ϐ‹‡†
‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻǤŠ‡–‡•–†‡•‹‰–‡…Š‹“—‡•†‡ϐ‹‡†‹–Š‹••–ƒ†ƒ”†…‘—Ž†„‡—•‡†
ˆ‘”–‡•–‹‰ƒ˜ƒ”‹‡–›‘ˆ–Š‡•‡…Šƒ”ƒ…–‡”‹•–‹…•ǤŠ‡”‡ƒ›„‡‘–Š‡”“—ƒŽ‹–›…Šƒ”ƒ…–‡”‹•–‹…•–Šƒ–…‘—Ž†„‡
…‘•‹†‡”‡†ȋ‡Ǥ‰Ǥ’”‹˜ƒ…›ȌǤ
Ȁ ʹͷͲͳͲ †‡ϐ‹‡• –Š‡ ’”‘†—…– “—ƒŽ‹–› ‘†‡Ž •Š‘™ „‡Ž‘™ ‹ ‹‰—”‡ Ǥͳǡ ™Š‹…Š …ƒ–‡‰‘”‹œ‡•
•›•–‡Ȁ•‘ˆ–™ƒ”‡ ’”‘†—…– “—ƒŽ‹–› ’”‘’‡”–‹‡• ‹–‘ ‡‹‰Š– …Šƒ”ƒ…–‡”‹•–‹…•ǣ ˆ—…–‹‘ƒŽ •—‹–ƒ„‹Ž‹–›ǡ
’‡”ˆ‘”ƒ…‡ ‡ˆϐ‹…‹‡…›ǡ …‘’ƒ–‹„‹Ž‹–›ǡ —•ƒ„‹Ž‹–›ǡ ”‡Ž‹ƒ„‹Ž‹–›ǡ •‡…—”‹–›ǡ ƒ‹–ƒ‹ƒ„‹Ž‹–› ƒ† ’‘”–ƒ„‹Ž‹–›Ǥ
ƒ…Š…Šƒ”ƒ…–‡”‹•–‹…‹•…‘’‘•‡†‘ˆƒ•‡–‘ˆ”‡Žƒ–‡†•—„Ǧ…Šƒ”ƒ…–‡”‹•–‹…•Ǥ•‘‡•‹–—ƒ–‹‘•ǡ–Š‡”‡ƒ›
„‡ ”‡‰—Žƒ–‘”› ”‡“—‹”‡‡–• ȋ‡Ǥ‰Ǥ ‰‘˜‡”‡– ’‘Ž‹…‹‡• ‘” Žƒ™•Ȍ –Šƒ– ƒ†ƒ–‡ –Šƒ– …‡”–ƒ‹ “—ƒŽ‹–›
…Šƒ”ƒ…–‡”‹•–‹…•ƒ”‡‡–„›ƒ•›•–‡Ǥƒ”‹‘—•–‡•–†‡•‹‰–‡…Š‹“—‡•ƒ†–›’‡•‘ˆ–‡•–‹‰…ƒ„‡—•‡†–‘
test for each characteristic (see Clauses A.3 and A.4).
Clause A.2’”‘˜‹†‡•ƒ‡š’Žƒƒ–‹‘‘ˆ–›’‡•‘ˆ–‡•–‹‰–Šƒ–…ƒ„‡—•‡†–‘–‡•––Š‡“—ƒŽ‹–›…Šƒ”ƒ…–‡”‹•–‹…•
presented in Figure A.1Ǥ  ƒ’’‹‰ ‘ˆ ‡ƒ…Š “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹… –‘ ƒ’’Ž‹…ƒ„Ž‡ –›’‡• ‘ˆ –‡•–‹‰ ‹•
presented in Clause A.3Ǥ Š‡ ”‡Žƒ–‹‘•Š‹’ „‡–™‡‡ –Š‡ “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹…• ƒ† –Š‡ •’‡…‹ϐ‹…ƒ–‹‘Ǧ
based and structure-based test design techniques covered in this part of ISO/IEC/IEEE 29119 is
explained in Clause A.4.

 Ȁ ʹͷͲ͵Ͳ ȋȀ ʹͷͲ͵ͲǣʹͲͲ͹Ȍ …ƒ „‡ —•‡† –‘ ‹†‡–‹ˆ› ƒ† †‘…—‡– •‘ˆ–™ƒ”‡ “—ƒŽ‹–›
”‡“—‹”‡‡–• –Šƒ– ƒ”‡ ƒ’’Ž‹…ƒ„Ž‡ –‘ ƒ –‡•– ‹–‡Ǥ Š‡•‡ …ƒ –Š‡ „‡ —•‡† –‘ ‹†‡–‹ˆ› “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹…•
‹ Ȁ ʹͷͲͳͲ ȋȀ ʹͷͲͳͲǣʹͲͳͳȌ ƒ† …‘””‡•’‘†‹‰ –›’‡• ‘ˆ –‡•–‹‰ –Šƒ– ƒ’’Ž› –‘ –‡•–‹‰ ‡ƒ…Š “—ƒŽ‹–›
requirement.

Figure A.1 — ISO/IEC 25010 product quality model

32 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

A.2 Quality-Related Types of Testing

A.2.1 Accessibility Testing
Š‡’—”’‘•‡‘ˆƒ……‡••‹„‹Ž‹–›–‡•–‹‰‹•–‘†‡–‡”‹‡™Š‡–Š‡”–Š‡–‡•–‹–‡…ƒ„‡‘’‡”ƒ–‡†„›—•‡”•™‹–Š
–Š‡™‹†‡•–”ƒ‰‡‘ˆ…Šƒ”ƒ…–‡”‹•–‹…•ƒ†…ƒ’ƒ„‹Ž‹–‹‡•ȋȀʹͷͲͳͲǣʹͲͳͳȌǡ‹…Ž—†‹‰•’‡…‹ϐ‹…ƒ……‡••‹„‹Ž‹–›
”‡“—‹”‡‡–•ȋ‡Ǥ‰Ǥ†—‡–‘ƒ‰‡ǡ˜‹•—ƒŽ‹’ƒ‹”‡–‘”Š‡ƒ”‹‰‹’ƒ‹”‡–ȌǤ……‡••‹„‹Ž‹–›–‡•–‹‰—•‡•ƒ
‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–•’‡…‹ϐ‹‡•‹–•ƒ……‡••‹„‹Ž‹–›”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰ƒ›ƒ……‡••‹„‹Ž‹–›†‡•‹‰
•–ƒ†ƒ”†• –‘ ™Š‹…Š –Š‡ –‡•– ‹–‡ —•– …‘ˆ‘”Ǥ ……‡••‹„‹Ž‹–› ”‡“—‹”‡‡–• ƒ”‡ …‘…‡”‡† ™‹–Š –Š‡
ƒ„‹Ž‹–›‘ˆƒ—•‡”™‹–Š•’‡…‹ϐ‹…ƒ……‡••‹„‹Ž‹–›‡‡†•–‘ƒ…Š‹‡˜‡ƒ……‡••‹„‹Ž‹–›‘„Œ‡…–‹˜‡•Ǥ ‘”‡šƒ’Ž‡ǡ–Š‹•
could include a requirement for the test item to support visual and/or hearing impaired users.
 Š‡‘”Ž†‹†‡‡„‘•‘”–‹—ȋ͵Ȍ†‡ϐ‹‡••–ƒ†ƒ”†•ˆ‘”ƒ……‡••‹„‹Ž‹–›ǡ‹…Ž—†‹‰ƒ……‡••‹„‹Ž‹–›‘ˆ
™‡„ƒ’’Ž‹…ƒ–‹‘•ƒ††‡˜‹…‡•Ǥ‹•‹–δhttp://www.w3.org/standards/> for more information.

A.2.2 Backup/Recovery Testing

Š‡ ’—”’‘•‡ ‘ˆ „ƒ…—’Ȁ”‡…‘˜‡”› –‡•–‹‰ ‹• –‘ †‡–‡”‹‡ ‹ˆǡ ‹ –Š‡ ‡˜‡– ‘ˆ ˆƒ‹Ž—”‡ǡ ƒ –‡•– ‹–‡ …ƒ „‡
”‡•–‘”‡†ˆ”‘„ƒ…—’–‘‹–•’”‡Ǧˆƒ‹Ž—”‡•–ƒ–‡Ǥƒ…—’Ȁ”‡…‘˜‡”›–‡•–‹‰—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–
•’‡…‹ϐ‹‡•‹–•„ƒ…—’ƒ†”‡…‘˜‡”›”‡“—‹”‡‡–•ǡ™Š‹…Š•’‡…‹ˆ›–Š‡‡‡†–‘„ƒ…—’–Š‡‘’‡”ƒ–‹‘ƒŽ•–ƒ–‡
‘ˆƒ–‡•–‹–‡ƒ–ƒ’‘‹–‹–‹‡ǡ‹…Ž—†‹‰†ƒ–ƒǡ…‘ϐ‹‰—”ƒ–‹‘ƒ†Ȁ‘”‡˜‹”‘‡–ƒ†”‡•–‘”‡–Š‡•–ƒ–‡
‘ˆ–Š‡–‡•–‹–‡ˆ”‘–Šƒ–„ƒ…—’Ǥƒ…—’Ȁ”‡…‘˜‡”›–‡•–‹‰–Š‡ˆ‘…—••‡•‘–‡•–‹‰–Š‡…‘””‡…–‡••‘ˆ
the test item’s backup and the correctness of the restored state of the test item against its pre-failure
•–ƒ–‡Ǥƒ…—’Ȁ”‡…‘˜‡”›–‡•–‹‰…ƒƒŽ•‘„‡—•‡†–‘˜‡”‹ˆ›™Š‡–Š‡”–Š‡„ƒ…—’ƒ†”‡…‘˜‡”›’”‘…‡†—”‡•
ˆ‘”–Š‡–‡•–‹–‡ƒ…Š‹‡˜‡•’‡…‹ϐ‹‡†”‡…‘˜‡”›‘„Œ‡…–‹˜‡•ǤŠ‹•–›’‡‘ˆ–‡•–‹‰ƒ›„‡…ƒ””‹‡†‘—–ƒ•’ƒ”–‘ˆ
ƒ†‹•ƒ•–‡””‡…‘˜‡”›–‡•–ȋ•‡‡A.2.5).

A.2.3 Compatibility Testing

Š‡’—”’‘•‡‘ˆ…‘’ƒ–‹„‹Ž‹–›–‡•–‹‰‹•–‘†‡–‡”‹‡™Š‡–Š‡”–Š‡–‡•–‹–‡…ƒˆ—…–‹‘ƒŽ‘‰•‹†‡‘–Š‡”
‹†‡’‡†‡–‘”†‡’‡†‡–ȋ„—–‘–‡…‡••ƒ”‹Ž›…‘—‹…ƒ–‹‰Ȍ’”‘†—…–•‹ƒ•Šƒ”‡†‡˜‹”‘‡–ȋ‹Ǥ‡Ǥ
…‘Ǧ‡š‹•–‡…‡ȌǤ‘’ƒ–‹„‹Ž‹–›–‡•–‹‰ƒ›ƒŽ•‘„‡ƒ’’Ž‹‡†–‘—Ž–‹’Ž‡…‘’‹‡•‘ˆ–Š‡•ƒ‡–‡•–‹–‡‘”–‘
multiple test items sharing a common environment.
‘’ƒ–‹„‹Ž‹–› ”‡“—‹”‡‡–• ˆ‘” –‡•– ‹–‡• –›’‹…ƒŽŽ› ‹…Ž—†‡ ‘‡ ‘” ‘”‡ ‘ˆ –Š‡ ˆ‘ŽŽ‘™‹‰ •—„Ǧ
requirements:
— Order of installation. Explicit order(s) of installation (otherwise it should be assumed that all possible
‘”†‡”•‘ˆ‹•–ƒŽŽƒ–‹‘ƒ”‡˜ƒŽ‹†Ȍ”‡•—Ž–•‹ƒ…‘ϐ‹‰—”ƒ–‹‘™Š‡”‡‡ƒ…Š–‡•–‹–‡™‹ŽŽ•—„•‡“—‡–Ž›
’‡”ˆ‘”‹–•”‡“—‹”‡†ˆ—…–‹‘•…‘””‡…–Ž›Ǥ
— Order of instantiation. Explicit order(s) of instantiation (otherwise it should be assumed that all
’‘••‹„Ž‡‘”†‡”•‘ˆ‹•–ƒ–‹ƒ–‹‘ƒ”‡˜ƒŽ‹†Ȍ”‡•—Ž–‹ƒ”—Ǧ–‹‡…‘ϐ‹‰—”ƒ–‹‘™Š‡”‡‡ƒ…Š–‡•–‹–‡
™‹ŽŽ•—„•‡“—‡–Ž›’‡”ˆ‘”‹–•”‡“—‹”‡†ˆ—…–‹‘•…‘””‡…–Ž›Ǥ
Ȅ ‘…—””‡– —•‡Ǥ Š‡ ƒ„‹Ž‹–› ‘ˆ –™‘ ‘” ‘”‡ –‡•– ‹–‡• –‘ ’‡”ˆ‘” –Š‡‹” ”‡“—‹”‡† ˆ—…–‹‘• ™Š‹Ž‡
”—‹‰ȋ„—–‘–‡…‡••ƒ”‹Ž›…‘—‹…ƒ–‹‰Ȍ‹–Š‡•ƒ‡‡˜‹”‘‡–Ǥ
Ȅ ˜‹”‘‡–…‘•–”ƒ‹–•Ǥ ‡ƒ–—”‡•‘ˆ–Š‡‡˜‹”‘‡–ǡ•—…Šƒ•‡‘”›ǡ’”‘…‡••‘”ǡƒ”…Š‹–‡…–—”‡ǡ
’Žƒ–ˆ‘” ‘” …‘ϐ‹‰—”ƒ–‹‘ǡ –Šƒ– ƒ› ƒˆˆ‡…– –Š‡ ƒ„‹Ž‹–› ‘ˆ –Š‡ –‡•– ‹–‡ –‘ ’‡”ˆ‘” ‹–• ”‡“—‹”‡†
ˆ—…–‹‘•…‘””‡…–Ž›Ǥ

A.2.4 Conversion Testing

The purpose of conversion testing is to determine whether data or software can continue to provide
”‡“—‹”‡†…ƒ’ƒ„‹Ž‹–‹‡•ƒˆ–‡”‘†‹ϐ‹…ƒ–‹‘•ƒ”‡ƒ†‡–‘–Š‡‹”ˆ‘”ƒ–ǡ•—…Šƒ•…‘˜‡”–‹‰ƒ’”‘‰”ƒˆ”‘
‘‡ ’”‘‰”ƒ‹‰ Žƒ‰—ƒ‰‡ –‘ ƒ‘–Š‡” ‘” …‘˜‡”–‹‰ ƒ ϐŽƒ– †ƒ–ƒ ϐ‹Ž‡ ‘” †ƒ–ƒ„ƒ•‡ ˆ”‘ ‘‡ ˆ‘”ƒ– –‘
ƒ‘–Š‡”Ǥ‡…‘‘•—„–›’‡‘ˆ…‘˜‡”•‹‘–‡•–‹‰‹•†ƒ–ƒ‹‰”ƒ–‹‘–‡•–‹‰Ǥ‘˜‡”•‹‘–‡•–‹‰—•‡•
ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–•’‡…‹ϐ‹‡•‹–•…‘˜‡”•‹‘”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰–Š‘•‡–Šƒ–—•–”‡ƒ‹

© ISO/IEC 2015 – All rights reserved 33

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

‹˜ƒ”‹ƒ––Š”‘—‰Š–Š‡…‘˜‡”•‹‘’”‘…‡••ǡ–Š‘•‡–Šƒ–ƒ”‡‡™ǡ‘†‹ϐ‹‡†ǡ‘”‘„•‘Ž‡–‡†„›–Š‡…‘˜‡”•‹‘
ƒ†ƒ›”‡“—‹”‡†…‘˜‡”•‹‘†‡•‹‰•–ƒ†ƒ”†•–‘™Š‹…Š–Š‡–‡•–‹–‡—•–…‘ˆ‘”Ǥ

A.2.5 Disaster Recovery Testing

Š‡’—”’‘•‡‘ˆ†‹•ƒ•–‡””‡…‘˜‡”›–‡•–‹‰‹•–‘†‡–‡”‹‡‹ˆǡ‹–Š‡‡˜‡–‘ˆˆƒ‹Ž—”‡ǡ‘’‡”ƒ–‹‘‘ˆ–Š‡–‡•–
item can be transferred to a different operating site and whether it can be transferred back again once
–Š‡ˆƒ‹Ž—”‡Šƒ•„‡‡”‡•‘Ž˜‡†Ǥ‹•ƒ•–‡””‡…‘˜‡”›–‡•–‹‰—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡ȋ–›’‹…ƒŽŽ›ƒ†‹•ƒ•–‡”
”‡…‘˜‡”› ’ŽƒȌ –Šƒ– •’‡…‹ϐ‹‡• ‹–• †‹•ƒ•–‡” ”‡…‘˜‡”› ”‡“—‹”‡‡–•ǡ ‹…Ž—†‹‰ ƒ› ”‡“—‹”‡† †‹•ƒ•–‡”
”‡…‘˜‡”› †‡•‹‰ •–ƒ†ƒ”†• –‘ ™Š‹…Š –Š‡ –‡•– ‹–‡ —•– …‘ˆ‘”Ǥ Š‡ –‡•– ‹–‡ ‹ †‹•ƒ•–‡” ”‡…‘˜‡”›
–‡•–‹‰ ƒ› „‡ ƒ ‡–‹”‡ ‘’‡”ƒ–‹‘ƒŽ •›•–‡ǡ ™‹–Š ƒ••‘…‹ƒ–‡† ˆƒ…‹Ž‹–‹‡•ǡ ’‡”•‘‡Žǡ ƒ† ’”‘…‡†—”‡•Ǥ
‹•ƒ•–‡””‡…‘˜‡”›–‡•–‹‰ƒ›…‘˜‡”ˆƒ…–‘”••—…Šƒ•’”‘…‡†—”‡•–‘„‡…ƒ””‹‡†‘—–„›‘’‡”ƒ–‹‘ƒŽ•–ƒˆˆǡ
”‡Ž‘…ƒ–‹‘‘ˆ†ƒ–ƒǡ•‘ˆ–™ƒ”‡ǡ’‡”•‘‡Žǡ‘ˆϐ‹…‡•ǡ‘”‘–Š‡”ˆƒ…‹Ž‹–‹‡•ǡ‘””‡…‘˜‡”‹‰†ƒ–ƒ’”‡˜‹‘—•Ž›„ƒ…‡†
up to a remote location.

A.2.6 Functional Testing

The purpose of functional testing is to determine whether the functional requirements of the test item
Šƒ˜‡„‡‡‡–Ǥ ‘”‡šƒ’Ž‡ǡ–Š‹•…‘—Ž†‹…Ž—†‡‹†‡–‹ˆ›‹‰™Š‡–Š‡”ƒˆ—…–‹‘Šƒ†„‡‡‹’Ž‡‡–‡†
ƒ……‘”†‹‰ –‘ ‹–• •’‡…‹ϐ‹‡† ”‡“—‹”‡‡–•Ǥ – …‘—Ž† „‡ …ƒ””‹‡† ‘—– —•‹‰ –Š‡ •’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† ƒ†
•–”—…–—”‡Ǧ„ƒ•‡†–‡•–†‡•‹‰–‡…Š‹“—‡•–Šƒ–ƒ”‡•’‡…‹ϐ‹‡†‹Clause 5.

A.2.7 Installability Testing

Š‡’—”’‘•‡‘ˆ‹•–ƒŽŽƒ„‹Ž‹–›–‡•–‹‰‹•–‘†‡–‡”‹‡‹ˆƒ–‡•–‹–‡ȋ•Ȍ…ƒ„‡‹•–ƒŽŽ‡†ǡ—‹•–ƒŽŽ‡†Ȁ”‡‘˜‡†
ƒ†Ȁ‘”—’‰”ƒ†‡†ƒ•”‡“—‹”‡†‹ ƒŽŽ •’‡…‹ϐ‹‡†‡˜‹”‘‡–•Ǥ•–ƒŽŽƒ„‹Ž‹–›–‡•–‹‰—•‡•ƒ ‘†‡Ž ‘ˆ–Š‡
‹•–ƒŽŽƒ„‹Ž‹–›”‡“—‹”‡‡–•‘ˆ–Š‡–‡•–‹–‡ǡ™Š‹…Šƒ”‡–›’‹…ƒŽŽ›•’‡…‹ϐ‹‡†‹–‡”•‘ˆ–Š‡‹•–ƒŽŽƒ–‹‘ǡ
uninstallation or upgrade processes (as described in the installation manual or guidelines), the people
™Š‘™‹ŽŽ…ƒ””›‘—––Š‡‹•–ƒŽŽƒ–‹‘ǡ—‹•–ƒŽŽƒ–‹‘‘”—’‰”ƒ†‡ǡ–Š‡–ƒ”‰‡–’Žƒ–ˆ‘”ȋ•Ȍƒ†–Š‡–‡•–‹–‡ȋ•Ȍ
to be installed, uninstalled or upgraded.

A.2.8 Interoperability Testing

Š‡’—”’‘•‡‘ˆ‹–‡”‘’‡”ƒ„‹Ž‹–›–‡•–‹‰‹•–‘†‡–‡”‹‡‹ˆƒ–‡•–‹–‡…ƒ‹–‡”ƒ…–…‘””‡…–Ž›™‹–Š‘–Š‡”
–‡•–‹–‡•‘”•›•–‡•‡‹–Š‡”‹–Š‡•ƒ‡‡˜‹”‘‡–‘”†‹ˆˆ‡”‡–‡˜‹”‘‡–•ǡ‹…Ž—†‹‰™Š‡–Š‡”–Š‡
–‡•–‹–‡…ƒƒ‡‡ˆˆ‡…–‹˜‡—•‡‘ˆ‹ˆ‘”ƒ–‹‘”‡…‡‹˜‡†ˆ”‘‘–Š‡”•›•–‡•Ǥ–‡”‘’‡”ƒ„‹Ž‹–›–‡•–‹‰
—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–•’‡…‹ϐ‹‡•‹–•‹–‡”‘’‡”ƒ„‹Ž‹–›”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰‹–‡”‘’‡”ƒ„‹Ž‹–›
design standards to which the test item must conform. This could include assessing whether a test
‹–‡”—‹‰‹‘‡‡˜‹”‘‡–…ƒ‹–‡”ƒ…–ƒ……—”ƒ–‡Ž›™‹–Šƒ‘–Š‡”–‡•–‹–‡‘”•›•–‡‹ƒ‘–Š‡”
separate environment.

A.2.9 Localization Testing

The purpose of localization testing is to determine whether the test item can be understood within the
geographical region it is required to be used in. Localization testing can include (but is not limited to)
ƒƒŽ›•‹•‘ˆ™Š‡–Š‡”–Š‡—•‡”‹–‡”ˆƒ…‡ƒ†•—’’‘”–‹‰†‘…—‡–ƒ–‹‘‘ˆ–Š‡–‡•–‹–‡…ƒ„‡—†‡”•–‘‘†
„›—•‡”•™‹–Š‹‡ƒ…Š…‘—–”›‘””‡‰‹‘‘ˆ—•‡Ǥ

A.2.10 Maintainability Testing

Š‡ ’—”’‘•‡ ‘ˆ ƒ‹–ƒ‹ƒ„‹Ž‹–› –‡•–‹‰ ‹• –‘ †‡–‡”‹‡ ‹ˆ ƒ –‡•– ‹–‡ …ƒ „‡ ƒ‹–ƒ‹‡† „› —•‹‰ ƒ
ƒ……‡’–ƒ„Ž‡ƒ‘—–‘ˆ‡ˆˆ‘”–Ǥƒ‹–ƒ‹ƒ„‹Ž‹–›–‡•–‹‰—•‡•ƒ‘†‡Ž‘ˆ–Š‡ƒ‹–ƒ‹ƒ„‹Ž‹–›”‡“—‹”‡‡–•
‘ˆ–Š‡–‡•–‹–‡ǡ™Š‹…Šƒ”‡–›’‹…ƒŽŽ›•’‡…‹ϐ‹‡†‹–‡”•‘ˆ–Š‡‡ˆˆ‘”–”‡“—‹”‡†–‘‡ˆˆ‡…–ƒ…Šƒ‰‡—†‡”–Š‡
following categories:
— corrective maintenance (i.e. correcting problems);
— perfective maintenance (i.e. enhancements);

34 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

— adaptive maintenance (i.e. adapting to changes in environment); and

— preventive maintenance (i.e. actions to reduce future maintenance costs).
ƒ‹–ƒ‹ƒ„‹Ž‹–›…ƒ„‡‹†‹”‡…–Ž›‡ƒ•—”‡†„›ƒ’’Ž›‹‰•–ƒ–‹…ƒƒŽ›•‹•Ǥ

A.2.11 Performance-Related Testing

Š‡ ’—”’‘•‡ ‘ˆ –Š‹• ˆƒ‹Ž› ‘ˆ –‡…Š‹“—‡• ‹• –‘ †‡–‡”‹‡ ™Š‡–Š‡” ƒ –‡•– ‹–‡ ’‡”ˆ‘”• ƒ• ”‡“—‹”‡†
™Š‡ ‹– ‹• ’Žƒ…‡† —†‡” ˜ƒ”‹‘—• –›’‡• ƒ† •‹œ‡• ‘ˆ DzŽ‘ƒ†dzǤ Š‹• ‹…Ž—†‡• ’‡”ˆ‘”ƒ…‡ǡ Ž‘ƒ†ǡ •–”‡••ǡ
‡†—”ƒ…‡ǡ˜‘Ž—‡ǡ…ƒ’ƒ…‹–›ƒ†‡‘”›ƒƒ‰‡‡––‡•–‹‰ǡ™Š‹…Š‡ƒ…Š—•‡ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡
–Šƒ–•’‡…‹ϐ‹‡•‹–•’‡”ˆ‘”ƒ…‡”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰ƒ›”‡“—‹”‡†’‡”ˆ‘”ƒ…‡†‡•‹‰•–ƒ†ƒ”†•–‘
™Š‹…Š–Š‡–‡•–‹–‡—•–…‘ˆ‘”Ǥ ‘”‡šƒ’Ž‡ǡ–Š‹•ƒ›‹…Ž—†‡ƒ••‡••‹‰–Š‡’‡”ˆ‘”ƒ…‡‘ˆ–Š‡–‡•–
item in terms of transactions per second, throughput response times, round trip time and resource
—–‹Ž‹œƒ–‹‘Ž‡˜‡Ž•ǤŠ‡Dz–›’‹…ƒŽdzŽ‘ƒ†‘ˆ–Š‡–‡•–‹–‡—†‡”Dz‘”ƒŽdz…‘†‹–‹‘•ƒ›„‡†‡ϐ‹‡†‹–Š‡
‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡‘ˆ–Š‡–‡•–‹–‡Ǥ
There are numerous techniques for assessing the performance of the test item:
— Performance testing is aimed at assessing the performance of the test item when it is placed under
ƒDz–›’‹…ƒŽdzŽ‘ƒ†Ǥ
Ȅ ‘ƒ†–‡•–‹‰‹•ƒ‹‡†ƒ–ƒ••‡••‹‰–Š‡„‡Šƒ˜‹‘—”‘ˆ–Š‡–‡•–‹–‡ȋ‡Ǥ‰Ǥ’‡”ˆ‘”ƒ…‡ƒ†”‡Ž‹ƒ„‹Ž‹–›Ȍ
™Š‡‹–‹•’Žƒ…‡†—†‡”…‘†‹–‹‘•‘ˆ˜ƒ”›‹‰Ž‘ƒ†•ǡ—•—ƒŽŽ›„‡–™‡‡ƒ–‹…‹’ƒ–‡†…‘†‹–‹‘•‘ˆŽ‘™ǡ
–›’‹…ƒŽƒ†’‡ƒ—•ƒ‰‡Ǥ
Ȅ –”‡••–‡•–‹‰‹•ƒ‹‡†ƒ–ƒ••‡••‹‰–Š‡’‡”ˆ‘”ƒ…‡‘ˆ–Š‡–‡•–‹–‡™Š‡‹–‹•’—•Š‡†„‡›‘†‹–•
ƒ–‹…‹’ƒ–‡†’‡ƒŽ‘ƒ†‘”™Š‡ƒ˜ƒ‹Žƒ„Ž‡”‡•‘—”…‡•ȋ‡Ǥ‰Ǥ‡‘”›ǡ’”‘…‡••‘”ǡ†‹•Ȍƒ”‡”‡†—…‡†„‡Ž‘™
•’‡…‹ϐ‹‡†‹‹—”‡“—‹”‡‡–•ǡ–‘‡˜ƒŽ—ƒ–‡Š‘™‹–„‡Šƒ˜‡•—†‡”‡š–”‡‡…‘†‹–‹‘•Ǥ
— Endurance testing (also called soak testing) is aimed at assessing whether the test item can sustain
the required load for a continuous period of time.
Ȅ ‘Ž—‡–‡•–‹‰‹•ƒ‹‡†ƒ–ƒ••‡••‹‰–Š‡’‡”ˆ‘”ƒ…‡‘ˆ–Š‡–‡•–‹–‡™Š‡‹–‹•’”‘…‡••‹‰•’‡…‹ϐ‹‡†
Ž‡˜‡Ž•‘ˆ†ƒ–ƒǤ ‘”‡šƒ’Ž‡ǡ–Š‹•ƒ›‹…Ž—†‡ƒ••‡••‹‰–‡•–‹–‡’‡”ˆ‘”ƒ…‡™Š‡‹–•†ƒ–ƒ„ƒ•‡‹•
‡ƒ”‹‰ƒš‹—…ƒ’ƒ…‹–›Ǥ
Ȅ ƒ’ƒ…‹–›–‡•–‹‰ȋƒŽ•‘…ƒŽŽ‡†•…ƒŽƒ„‹Ž‹–›–‡•–‹‰Ȍ‹•ƒ‹‡†ƒ–ƒ••‡••‹‰Š‘™–Š‡–‡•–‹–‡™‹ŽŽ’‡”ˆ‘”
—†‡” …‘†‹–‹‘• –Šƒ– ƒ› ‡‡† –‘ „‡ •—’’‘”–‡† ‹ –Š‡ ˆ—–—”‡Ǥ ‘” ‡šƒ’Ž‡ǡ –Š‹• ƒ› ‹…Ž—†‡
ƒ••‡••‹‰™Šƒ–Ž‡˜‡Ž‘ˆƒ††‹–‹‘ƒŽ”‡•‘—”…‡•ȋ‡Ǥ‰Ǥ‡‘”›ǡ†‹•…ƒ’ƒ…‹–›ǡ‡–™‘”„ƒ†™‹†–ŠȌ™‹ŽŽ
be required to support anticipated future loads.
Ȅ ‡‘”›ƒƒ‰‡‡––‡•–‹‰‹•ƒ‹‡†ƒ–ƒ••‡••‹‰Š‘™–Š‡–‡•–‹–‡™‹ŽŽ’‡”ˆ‘”‹–‡”•‘ˆ–Š‡
ƒ‘—–ȋ‘”ƒŽŽ›ƒš‹—Ȍ‘ˆ‡‘”›—•‡†ȋ‡Ǥ‰ǤŠƒ”††‹•‡‘”›ǡƒ†Ȍǡ–Š‡–›’‡‘ˆ
‡‘”› ȋ‡Ǥ‰Ǥ †›ƒ‹… ‘” ƒŽŽ‘…ƒ–‡†Ȁ•–ƒ–‹…Ȍ ƒ†Ȁ‘” †‡ϐ‹‡† Ž‡˜‡Ž• ‘ˆ ‡‘”› Ž‡ƒƒ‰‡ ‡š’‡”‹‡…‡†
†—”‹‰ –‡•–‹‰Ǥ ‡‘”› ”‡“—‹”‡‡–• ™‹ŽŽ –›’‹…ƒŽŽ› „‡ •’‡…‹ϐ‹‡† ‹ –‡”• ‘ˆ •’‡…‹ϐ‹… ‘’‡”ƒ–‹‰
…‘†‹–‹‘•ȋ‡Ǥ‰Ǥƒ’‡ƒ‡‘”›”‡“—‹”‡‡–‘˜‡”ƒ’ƒ”–‹…—Žƒ”’‡”‹‘†‘ˆ‘’‡”ƒ–‹‘—†‡”†‡ϐ‹‡†
–”ƒ•ƒ…–‹‘Ž‘ƒ†•ƒ›„‡•’‡…‹ϐ‹‡†ȌǤ

A.2.12 Portability Testing

Š‡ ’—”’‘•‡ ‘ˆ ’‘”–ƒ„‹Ž‹–› –‡•–‹‰ ‹• –‘ †‡–‡”‹‡ –Š‡ †‡‰”‡‡ ‘ˆ ‡ƒ•‡ ‘” †‹ˆϐ‹…—Ž–› –‘ ™Š‹…Š ƒ –‡•–
‹–‡…ƒ„‡‡ˆˆ‡…–‹˜‡Ž›ƒ†‡ˆϐ‹…‹‡–Ž›–”ƒ•ˆ‡””‡†ˆ”‘‘‡Šƒ”†™ƒ”‡ǡ•‘ˆ–™ƒ”‡‘”‘–Š‡”‘’‡”ƒ–‹‘ƒŽ
‘” —•ƒ‰‡ ‡˜‹”‘‡– –‘ ƒ‘–Š‡”Ǥ ‘”–ƒ„‹Ž‹–› –‡•–‹‰ —•‡• ƒ ‘†‡Ž ‘ˆ –Š‡ –‡•– ‹–‡ –Šƒ– •’‡…‹ϐ‹‡• ‹–•
’‘”–ƒ„‹Ž‹–›”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰ƒ›”‡“—‹”‡†’‘”–ƒ„‹Ž‹–›†‡•‹‰•–ƒ†ƒ”†•–‘™Š‹…Š–Š‡–‡•–‹–‡
—•–…‘ˆ‘”Ǥ‘”–ƒ„‹Ž‹–›”‡“—‹”‡‡–•ƒ”‡…‘…‡”‡†™‹–Š–Š‡ƒ„‹Ž‹–›–‘–”ƒ•ˆ‡”–Š‡–‡•–‹–‡ˆ”‘
‘‡‡˜‹”‘‡––‘ƒ‘–Š‡”ǡ‘”–‘ƒŽ–‡”–Š‡…‘ϐ‹‰—”ƒ–‹‘‘ˆ–Š‡‡š‹•–‹‰‡˜‹”‘‡––‘‘–Š‡””‡“—‹”‡†
…‘ϐ‹‰—”ƒ–‹‘•Ǥ ‘”‡šƒ’Ž‡ǡ–Š‹•…‘—Ž†‹…Ž—†‡ƒ••‡••‹‰™Š‡–Š‡”–Š‡–‡•–‹–‡…ƒ„‡‘’‡”ƒ–‡†ˆ”‘ƒ
˜ƒ”‹‡–›‘ˆ†‹ˆˆ‡”‡–„”‘™•‡”•Ǥ

© ISO/IEC 2015 – All rights reserved 35

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

A.2.13 Procedure Testing

The purpose of procedure testing is to determine whether procedural instructions meet user
requirements and support the purpose of their use. Procedure testing uses a model of the procedural
”‡“—‹”‡‡–•‘ˆ–Š‡–‡•–‹–‡ƒ•ƒ…‘’Ž‡–‡ƒ††‡Ž‹˜‡”‡†—‹–Ǥ”‘…‡†—”‡”‡“—‹”‡‡–•†‡ϐ‹‡™Šƒ–
‹•‡š’‡…–‡†‘ˆƒ›’”‘…‡†—”ƒŽ†‘…—‡–ƒ–‹‘ƒ†ƒ”‡™”‹––‡‹–Š‡ˆ‘”‘ˆ’”‘…‡†—”ƒŽ‹•–”—…–‹‘•Ǥ
Š‡•‡’”‘…‡†—”ƒŽ‹•–”—…–‹‘•™‹ŽŽ‘”ƒŽŽ›…‘‡‹–Š‡ˆ‘”‘ˆ‘‡‘ˆ–Š‡ˆ‘ŽŽ‘™‹‰†‘…—‡–•ǣ
— a user guide;
— an instruction manual;
— a user reference manual.
Š‹•‹ˆ‘”ƒ–‹‘™‹ŽŽ‘”ƒŽŽ›†‡ϐ‹‡Š‘™–Š‡—•‡”‹•‡ƒ––‘ǣ
— set up the test item for normal usage;
— operate the test item in normal conditions;
Ȅ „‡…‘‡ƒ…‘’‡–‡–—•‡”‘ˆ–Š‡•›•–‡ȋ–—–‘”‹ƒŽϐ‹Ž‡•ȌǢ
— trouble-shoot the test item when faults arise;
Ȅ ”‡…‘ϐ‹‰—”‡–Š‡–‡•–‹–‡Ǥ

A.2.14 Reliability Testing

Š‡ ’—”’‘•‡ ‘ˆ ”‡Ž‹ƒ„‹Ž‹–› –‡•–‹‰ ‹• –‘ ‡˜ƒŽ—ƒ–‡ –Š‡ ƒ„‹Ž‹–› ‘ˆ –Š‡ –‡•– ‹–‡ –‘ ’‡”ˆ‘” ‹–• ”‡“—‹”‡†
ˆ—…–‹‘•ǡ‹…Ž—†‹‰‡˜ƒŽ—ƒ–‹‰–Š‡ˆ”‡“—‡…›™‹–Š™Š‹…Šˆƒ‹Ž—”‡•‘……—”ǡ™Š‡‹–‹•—•‡†—†‡”•–ƒ–‡†
…‘†‹–‹‘•ˆ‘”ƒ•’‡…‹ϐ‹‡†’‡”‹‘†‘ˆ–‹‡Ǥ‡Ž‹ƒ„‹Ž‹–›–‡•–‹‰—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–•’‡…‹ϐ‹‡•
‹–•”‡“—‹”‡†Ž‡˜‡Ž‘ˆ”‡Ž‹ƒ„‹Ž‹–›ȋ‡Ǥ‰Ǥ‡ƒ–‹‡–‘ˆƒ‹Ž—”‡ǡ‡ƒ–‹‡„‡–™‡‡ˆƒ‹Ž—”‡•ȌǤŠ‡‘†‡Ž•Š‘—Ž†
‹…Ž—†‡ƒ†‡ϐ‹‹–‹‘‘ˆˆƒ‹Ž—”‡ƒ†‡‹–Š‡”–Š‡‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡‘ˆ–Š‡–‡•–‹–‡‘”ƒƒ’’”‘ƒ…Š–‘†‡”‹˜‡
–Š‡‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡Ǥ

A.2.15 Security Testing

Š‡’—”’‘•‡‘ˆ•‡…—”‹–›–‡•–‹‰‹•–‘‡˜ƒŽ—ƒ–‡–Š‡†‡‰”‡‡–‘™Š‹…Šƒ–‡•–‹–‡ƒ†‹–•ƒ••‘…‹ƒ–‡††ƒ–ƒ
ƒ”‡’”‘–‡…–‡†•‘–Šƒ–—ƒ—–Š‘”‹•‡†’‡”•‘•‘”•›•–‡•…ƒ‘–—•‡ǡ”‡ƒ†‘”‘†‹ˆ›–Š‡ƒ†ƒ—–Š‘”‹•‡†
’‡”•‘•‘”•›•–‡•ƒ”‡‰”ƒ–‡†”‡“—‹”‡†ƒ……‡••–‘–Š‡Ǥ‡…—”‹–›–‡•–‹‰—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡
–Šƒ–•’‡…‹ϐ‹‡•‹–••‡…—”‹–›”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰ƒ›”‡“—‹”‡†•‡…—”‹–›†‡•‹‰•–ƒ†ƒ”†•–‘™Š‹…Š–Š‡
–‡•–‹–‡—•–…‘ˆ‘”Ǥ‡…—”‹–›”‡“—‹”‡‡–•ƒ”‡…‘…‡”‡†™‹–Š–Š‡ƒ„‹Ž‹–›–‘’”‘–‡…––Š‡†ƒ–ƒƒ†
ˆ—…–‹‘ƒŽ‹–›‘ˆƒ–‡•–‹–‡ˆ”‘—ƒ—–Š‘”‹•‡†—•‡”•ƒ†ƒŽ‹…‹‘—•—•‡Ǥ ‘”‡šƒ’Ž‡ǡ–Š‹•…‘—Ž†‹…Ž—†‡
assessing whether the test item prevents unauthorised users from accessing data, or whether certain
ˆ—…–‹‘•‘ˆƒ–‡•–‹–‡–Šƒ–ƒ”‡‘Ž›”‡“—‹”‡†–‘„‡ƒ……‡••‹„Ž‡„›…‡”–ƒ‹—•‡”–›’‡•ƒ”‡’”‘–‡…–‡†ˆ”‘
‘–Š‡”—•‡”–›’‡•Ǥ
Š‡”‡ƒ”‡ƒ—„‡”‘ˆ–‡…Š‹“—‡•ˆ‘”ƒ••‡••‹‰–Š‡•‡…—”‹–›‘ˆƒ–‡•–‹–‡ǣ
Ȅ ‡‡–”ƒ–‹‘ –‡•–‹‰ ‹˜‘Ž˜‡• ƒ––‡’–‡† ƒ……‡•• –‘ ƒ –‡•– ‹–‡ ȋ‹…Ž—†‹‰ ‹–• ˆ—…–‹‘ƒŽ‹–› ƒ†Ȁ‘”
’”‹˜ƒ–‡†ƒ–ƒȌ„›ƒ–‡•–‡”–Šƒ–‹•‹‹…‹‰–Š‡ƒ…–‹‘•‘ˆƒ—ƒ—–Š‘”‹•‡†—•‡”Ǥ
Ȅ ”‹˜ƒ…› –‡•–‹‰ ‹˜‘Ž˜‡• ƒ––‡’–‡† ƒ……‡•• –‘ ’”‹˜ƒ–‡ †ƒ–ƒ ƒ† ˜‡”‹ϐ‹…ƒ–‹‘ ‘ˆ –Š‡ ƒ—†‹– –”ƒ‹Ž ȋ‹Ǥ‡Ǥ
trace) that is left behind when users access private data.
Ȅ ‡…—”‹–›ƒ—†‹–‹‰‹•ƒ–›’‡‘ˆ•–ƒ–‹…–‡•–‹‰‹™Š‹…Šƒ–‡•–‡”‹•’‡…–•ǡ”‡˜‹‡™•‘”™ƒŽ•–Š”‘—‰Š–Š‡
”‡“—‹”‡‡–•ƒ†…‘†‡‘ˆƒ–‡•–‹–‡–‘†‡–‡”‹‡™Š‡–Š‡”ƒ›•‡…—”‹–›˜—Ž‡”ƒ„‹Ž‹–‹‡•ƒ”‡’”‡•‡–Ǥ
Ȅ —Ž‡”ƒ„‹Ž‹–›•…ƒ‹‰‹˜‘Ž˜‡•–Š‡—•‡‘ˆƒ—–‘ƒ–‡†–‡•–‹‰–‘‘Ž•–‘•…ƒƒ–‡•–‹–‡ˆ‘”•‹‰•‘ˆ
•’‡…‹ϐ‹…‘™˜—Ž‡”ƒ„‹Ž‹–‹‡•Ǥ

36 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

A.2.16 Usability Testing

Š‡’—”’‘•‡‘ˆ—•ƒ„‹Ž‹–›–‡•–‹‰‹•–‘‡˜ƒŽ—ƒ–‡™Š‡–Š‡”•’‡…‹ϐ‹‡†—•‡”•…ƒ—•‡–Š‡–‡•–‹–‡–‘ƒ…Š‹‡˜‡
ƒ••‹‰‡† ‰‘ƒŽ• ™‹–Š ‡ˆˆ‡…–‹˜‡‡••ǡ ‡ˆϐ‹…‹‡…› ƒ† •ƒ–‹•ˆƒ…–‹‘ ‹ •’‡…‹ϐ‹‡† …‘–‡š–• ‘ˆ —•‡Ǥ •ƒ„‹Ž‹–›
–‡•–‹‰–Š‡”‡ˆ‘”‡—•‡•ƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡–Šƒ–•’‡…‹ϐ‹‡•‹–•—•ƒ„‹Ž‹–›”‡“—‹”‡‡–•ǡ‹…Ž—†‹‰ƒ›
”‡“—‹”‡†—•ƒ„‹Ž‹–›†‡•‹‰•–ƒ†ƒ”†•–‘™Š‹…Š–Š‡–‡•–‹–‡—•–…‘ˆ‘”Ǥ•ƒ„‹Ž‹–›”‡“—‹”‡‡–••’‡…‹ˆ›
–Š‡—•ƒ„‹Ž‹–›‰‘ƒŽ•ˆ‘”–Š‡–‡•–‹–‡Ǥ•ƒ„‹Ž‹–›‰‘ƒŽ•—•–„‡ „ƒ•‡†‘–‡•–‹–‡‰‘ƒŽ•ȋ–Š‡”‡ƒ•‘ˆ‘”
Šƒ˜‹‰–Š‡–‡•–‹–‡ǡ–Š‡†‹ˆˆ‡”‡…‡‹–‹•–‘„”‹‰ƒ„‘—–ˆ‘”–Š‡‘”‰ƒ‹œƒ–‹‘‘”‹†‹˜‹†—ƒŽǡ–Š‡—•ƒ„‹Ž‹–›Ǧ
related purpose and tasks it will aid), and the contexts of use for the test item (who is to use the test
‹–‡ƒ†–Š‡‡˜‹”‘‡–‹™Š‹…Š‹–‹•–‘„‡—•‡†ǡ—•‡”…Šƒ”ƒ…–‡”‹•–‹…•ƒ†—•‡”–ƒ••ȌǤ•ƒ„‹Ž‹–›‰‘ƒŽ•
™‹ŽŽ„‡†‡ϐ‹‡†ˆ‘”–Š‡‡ˆˆ‡…–‹˜‡‡••ǡ‡ˆϐ‹…‹‡…›ƒ†•ƒ–‹•ˆƒ…–‹‘ˆ‘”•’‡…‹ϐ‹‡†—•‡”•–‘ƒ…Š‹‡˜‡•’‡…‹ϐ‹‡†
‰‘ƒŽ•‹‘‡‘”‘”‡•’‡…‹ϐ‹‡†…‘–‡š–•‘ˆ—•‡Ǥ
 ȀͻʹͶͳ†‡ϐ‹‡••–ƒ†ƒ”†•ˆ‘”†‡ϐ‹‹‰–Š‡”‡“—‹”‡‡–•ˆ‘”Š—ƒǦ•›•–‡•‹–‡”ƒ…–‹‘Ǥ

A.3 Mapping Quality Characteristics to Types of Testing

A.3.1 Mapping
 –Š‡ –ƒ„Ž‡ „‡Ž‘™ǡ –Š‡ –›’‡• ‘ˆ –‡•–‹‰ –Šƒ– ™‡”‡ ’”‡•‡–‡† ‹ Clause A.2 ƒ”‡ ƒ’’‡† –‘ –Š‡ “—ƒŽ‹–›
characteristics that were presented in Figure A.1 (in A.1.1).

Table A.1 — Mapping of ISO/IEC 25010 product quality characteristics to types of testing

Type of Testing Quality Characteristic Sub-Characteristics

……‡••‹„‹Ž‹–›‡•–‹‰ •ƒ„‹Ž‹–› ……‡••‹„‹Ž‹–›
ƒ…—’Ȁ‡…‘˜‡”›‡•–‹‰ ‡Ž‹ƒ„‹Ž‹–› ƒ–—”‹–›
Fault tolerance
‡…‘˜‡”ƒ„‹Ž‹–›
‘’ƒ–‹„‹Ž‹–›‡•–‹‰ ‘’ƒ–‹„‹Ž‹–› Co-existence
Conversion Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‹•ƒ•–‡”‡…‘˜‡”›‡•–‹‰ ‡Ž‹ƒ„‹Ž‹–› ƒ–—”‹–›
Fault tolerance
‡…‘˜‡”ƒ„‹Ž‹–›
Functional Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•–ƒŽŽƒ„‹Ž‹–›‡•–‹‰ ‘”–ƒ„‹Ž‹–› •–ƒŽŽƒ„‹Ž‹–›
–‡”‘’‡”ƒ„‹Ž‹–›‡•–‹‰ ‘’ƒ–‹„‹Ž‹–› –‡”‘’‡”ƒ„‹Ž‹–›

© ISO/IEC 2015 – All rights reserved 37

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table A.1 (continued)

Type of Testing Quality Characteristic Sub-Characteristics
Localization Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› ’’”‘’”‹ƒ–‡‡••”‡…‘‰‹•ƒ„‹Ž‹–›
‡ƒ”ƒ„‹Ž‹–›
’‡”ƒ„‹Ž‹–›
User error protection
User interface aesthetics
……‡••‹„‹Ž‹–›
‘”–ƒ„‹Ž‹–› †ƒ’–ƒ„‹Ž‹–›
ƒ‹–ƒ‹ƒ„‹Ž‹–›‡•–‹‰ ƒ‹–ƒ‹ƒ„‹Ž‹–› ‘†—Žƒ”‹–›
‡—•ƒ„‹Ž‹–›
ƒŽ›•ƒ„‹Ž‹–›
‘†‹ϐ‹ƒ„‹Ž‹–›
‡•–ƒ„‹Ž‹–›
Performance-Related Testing ‡”ˆ‘”ƒ…‡‡ˆϐ‹…‹‡…› Time-behaviour
Resource utilisation
ƒ’ƒ…‹–›
‘”–ƒ„‹Ž‹–›‡•–‹‰ ‘”–ƒ„‹Ž‹–› †ƒ’–ƒ„‹Ž‹–›
•–ƒŽŽƒ„‹Ž‹–›
‡’Žƒ…‡ƒ„‹Ž‹–›
Procedure Testing None None
‡Ž‹ƒ„‹Ž‹–›‡•–‹‰ ‡Ž‹ƒ„‹Ž‹–› ƒ–—”‹–›
˜ƒ‹Žƒ„‹Ž‹–›
Fault tolerance
‡…‘˜‡”ƒ„‹Ž‹–›
‡…—”‹–›‡•–‹‰ ‡…—”‹–› ‘ϐ‹†‡–‹ƒŽ‹–›
–‡‰”‹–›
Non-repudiation
……‘—–ƒ„‹Ž‹–›
—–Š‡–‹…‹–›
•ƒ„‹Ž‹–›‡•–‹‰ •ƒ„‹Ž‹–› ’’”‘’”‹ƒ–‡‡••”‡…‘‰‹•ƒ„‹Ž‹–›
‡ƒ”ƒ„‹Ž‹–›
’‡”ƒ„‹Ž‹–›
User error protection
User interface aesthetics
……‡••‹„‹Ž‹–›

38 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

A.4 Mapping Quality Characteristics to Test Design Techniques

A.4.1 Mapping
Š‡–‡•–†‡•‹‰–‡…Š‹“—‡•†‡•…”‹„‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ…ƒ„‡—•‡†–‘–‡•–ƒ˜ƒ”‹‡–›
‘ˆ –Š‡ “—ƒŽ‹–› …Šƒ”ƒ…–‡”‹•–‹…• Ž‹•–‡† ‹ Figure A.1. The following table provides an example mapping
between them.

Table A.2 — Mapping of test design techniques to product quality measures for ISO/IEC 25010
product characteristics

Test Design Technique ISO/IEC 25010 Quality Characteristic ISO/IEC 25010 Sub-Characteristics
’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡•
‘—†ƒ”›ƒŽ—‡ƒŽ›•‹• —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‡”ˆ‘”ƒ…‡ˆϐ‹…‹‡…› Time-behaviour
ƒ’ƒ…‹–›
•ƒ„‹Ž‹–› User error protection
‡Ž‹ƒ„‹Ž‹–› Fault tolerance
‡…—”‹–› ‘ϐ‹†‡–‹ƒŽ‹–›
–‡‰”‹–›
Cause-Effect Graphing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› User error protection
‘’ƒ–‹„‹Ž‹–› Co-existence
Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘† —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› User error protection
Combinatorial Test Design —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Techniques
Functional correctness
Functional appropriateness
‘’ƒ–‹„‹Ž‹–› Co-existence
‡”ˆ‘”ƒ…‡ˆϐ‹…‹‡…› Time-behaviour
•ƒ„‹Ž‹–› User error protection
Decision Table Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‘’ƒ–‹„‹Ž‹–› Co-existence
•ƒ„‹Ž‹–› User error protection

© ISO/IEC 2015 – All rights reserved 39

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)
Test Design Technique
Equivalence Partitioning
Table A.2 (continued)
ISO/IEC 25010 Quality Characteristic ISO/IEC 25010 Sub-Characteristics
—…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› User error protection
‡Ž‹ƒ„‹Ž‹–› ˜ƒ‹Žƒ„‹Ž‹–›
‡…—”‹–› ‘ϐ‹†‡–‹ƒŽ‹–›
–‡‰”‹–›
Non-repudiation
……‘—–ƒ„‹Ž‹–›
—–Š‡–‹…‹–›
Random Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Performance Time behaviour
Resource utilisation
ƒ’ƒ…‹–›
‡Ž‹ƒ„‹Ž‹–› ƒ–—”‹–›
˜ƒ‹Žƒ„‹Ž‹–›
Fault tolerance
‡…‘˜‡”ƒ„‹Ž‹–›
‡…—”‹–› ‘ϐ‹†‡–‹ƒŽ‹–›
–‡‰”‹–›
Non-repudiation
……‘—–ƒ„‹Ž‹–›
—–Š‡–‹…‹–›
Scenario Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› ‡ƒ”ƒ„‹Ž‹–›
’‡”ƒ„‹Ž‹–›
User error protection
User interface aesthetics
……‡••‹„‹Ž‹–›
’’”‘’”‹ƒ–‡‡••”‡…‘‰‹•ƒ„‹Ž‹–›
State Transition Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‡Ž‹ƒ„‹Ž‹–› ƒ–—”‹–›
˜ƒ‹Žƒ„‹Ž‹–›
Fault tolerance
‡…‘˜‡”ƒ„‹Ž‹–›

40 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table A.2 (continued)

Test Design Technique ISO/IEC 25010 Quality Characteristic ISO/IEC 25010 Sub-Characteristics
›–ƒš‡•–‹‰ —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Use Case Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
•ƒ„‹Ž‹–› ‡ƒ”ƒ„‹Ž‹–›
’‡”ƒ„‹Ž‹–›
User error protection
User interface aesthetics
……‡••‹„‹Ž‹–›
’’”‘’”‹ƒ–‡‡••”‡…‘‰‹•ƒ„‹Ž‹–›
Structure-Based Test Design Techniques
Branch Condition Combina- —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
tion Testing
Functional correctness
Functional appropriateness
Branch Condition Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Branch Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Data Flow Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Decision Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘ —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
‘˜‡”ƒ‰‡ȋȌ‡•–‹‰
Functional correctness
Functional appropriateness
Statement Testing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
Experience-Based Test Design Techniques

© ISO/IEC 2015 – All rights reserved 41

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table A.2 (continued)

Test Design Technique ISO/IEC 25010 Quality Characteristic ISO/IEC 25010 Sub-Characteristics
Error Guessing —…–‹‘ƒŽ—‹–ƒ„‹Ž‹–› Functional completeness
Functional correctness
Functional appropriateness
‡”ˆ‘”ƒ…‡ˆϐ‹…‹‡…› Time-behaviour
Resource utilisation
ƒ’ƒ…‹–›
•ƒ„‹Ž‹–› ‡ƒ”ƒ„‹Ž‹–›
’‡”ƒ„‹Ž‹–›
User error protection
‡Ž‹ƒ„‹Ž‹–› Fault tolerance

42 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Annex B
(informative)

—‹†‡Ž‹‡•ƒ†šƒ’Ž‡•ˆ‘”–Š‡’’Ž‹…ƒ–‹‘‘ˆ’‡…‹ϐ‹…ƒ–‹‘Ǧ
Based Test Design Techniques

Ǥͳ
—‹†‡Ž‹‡•ƒ†šƒ’Ž‡•ˆ‘”’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‹‰
B.1.1 Overview
This annex provides guidance on the requirements in 5.2 and 6.2„›†‡‘•–”ƒ–‹‰–Š‡ƒ’’Ž‹…ƒ–‹‘‘ˆ
‡ƒ…Š‹†‹˜‹†—ƒŽ•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†–‡•–†‡•‹‰–‡…Š‹“—‡–‘ƒ•‡’ƒ”ƒ–‡’”‘„Ž‡Ǥƒ…Š‡šƒ’Ž‡ˆ‘ŽŽ‘™•
–Š‡‡•–‡•‹‰ƒ†’Ž‡‡–ƒ–‹‘”‘…‡••–Šƒ–‹•†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦʹǤŽ–Š‘—‰Š‡ƒ…Š
‡šƒ’Ž‡ ‹• ƒ’’Ž‹‡† ‹ ƒ •’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† –‡•–‹‰ …‘–‡š–ǡ ƒ• •–ƒ–‡† ‹ 5.1, in practice most of the
–‡…Š‹“—‡•†‡ϐ‹‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ…ƒ„‡—•‡†‹–‡”…Šƒ‰‡ƒ„Ž›ȋ‡Ǥ‰Ǥ„‘—†ƒ”›˜ƒŽ—‡
ƒƒŽ›•‹•…‘—Ž†„‡—•‡†–‘–‡•––Š‡‹’—–•–‘ƒ’”‘‰”ƒ–Š”‘—‰Š–Š‡—•‡”‹–‡”ˆƒ…‡‘”–Š‡„‘—†ƒ”‹‡•‘ˆ
variables within program source code).

Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡•–‡•‹‰‡…Š‹“—‡šƒ’Ž‡•
B.2.1 Equivalence Partitioning

B.2.1.1 Introduction

The aim of equivalence partitioning is to derive a set of test cases that cover the input and output
partitions of the test item according to the chosen level of equivalence partition coverage. Equivalence
partitioning is based on the premise that the inputs and outputs of a test item can be partitioned into
…Žƒ••‡•–Šƒ–ǡƒ……‘”†‹‰–‘–Š‡–‡•–„ƒ•‹•ˆ‘”–Š‡–‡•–‹–‡ǡ™‹ŽŽ„‡–”‡ƒ–‡†•‹‹Žƒ”Ž›„›–Š‡–‡•–‹–‡ǤŠ—•
–Š‡ ”‡•—Ž– ‘ˆ –‡•–‹‰ ƒ› ˜ƒŽ—‡ ‹ ‘‡ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘ …ƒ „‡ …‘•‹†‡”‡† ”‡’”‡•‡–ƒ–‹˜‡ ‘ˆ –Š‡
”‡•—Ž–‘ˆ–‡•–‹‰™‹–Šƒ›‘–Š‡”˜ƒŽ—‡‹–Šƒ–’ƒ”–‹–‹‘Ǥ

ǤʹǤͳǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider a test item, generate_grading, with the following test basis:

The component receives an exam mark (out of 75) and a coursework (c/w) mark (out of 25) as input,
from which it outputs a grade for the course in the range ‘A’ to ‘D’. The grade is generated by calculating
the overall mark, which is the sum of the exam and c/w marks, as follows:

greater than or equal to 70 - ‘A’

greater than or equal to 50, but less than 70 - ‘B’

greater than or equal to 30, but less than 50 - ‘C’

less than 30 - ‘D’

Where invalid input(s) are detected (e.g. a mark is outside its expected range) then a fault message
(‘FM’) is generated. All inputs are passed as integers.

B.2.1.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
© ISO/IEC 2015 – All rights reserved 43
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

ͳǣ‰‡‡”ƒ–‡̴‰”ƒ†‹‰ˆ—…–‹‘

B.2.1.4 Step 2: Derive Test Conditions (TD2)

In equivalence partitioning, the equivalence partitions are test conditions (TCOND). Equivalence
’ƒ”–‹–‹‘•ƒ”‡‹†‡–‹ϐ‹‡†ˆ”‘„‘–Š–Š‡‹’—–•ƒ†‘—–’—–•‘ˆ–Š‡–‡•–‹–‡ǤƒŽ‹†ƒ†‹˜ƒŽ‹†‹’—–•ƒ†
outputs are considered.
Š‡’ƒ”–‹–‹‘•ˆ‘”–Š‡–™‘‹’—–•ƒ”‡‹‹–‹ƒŽŽ›‹†‡–‹ϐ‹‡†ǤŠ‡valid’ƒ”–‹–‹‘•…ƒ„‡†‡•…”‹„‡†„›ǣ

TCOND1: Ͳδα‡šƒƒ”δα͹ͷ (for FS1)

TCOND2: Ͳδα…‘—”•‡™‘”ƒ”δαʹͷ (for FS1)

The most obvious invalid’ƒ”–‹–‹‘•„ƒ•‡†‘–Š‡‹’—–•…ƒ„‡†‡•…”‹„‡†„›ǣ

TCOND3: ‡šƒƒ”δͲ (for FS1)

TCOND4: exam mark > 75 (for FS1)

TCOND5: …‘—”•‡™‘”ƒ”δͲ (for FS1)

TCOND6: coursework mark > 25 (for FS1)

ƒ”–‹–‹‘‡†”ƒ‰‡•‘ˆ˜ƒŽ—‡•…ƒ„‡”‡’”‡•‡–‡†’‹…–‘”‹ƒŽŽ›ǡ–Š‡”‡ˆ‘”‡ǡˆ‘”–Š‡‹’—–ǡ‡šƒƒ”ǡ™‡‰‡–ǣ

Figure B.1 — Input “exam mark”

And for the input, coursework mark, we get:

Figure B.2 — Input “coursework mark”

‡••‘„˜‹‘—•‹˜ƒŽ‹†‹’—–’ƒ”–‹–‹‘•…‘—Ž†‹…Ž—†‡ƒ›‘–Š‡”‹’—––›’‡•ǡ•—…Šƒ•‘Ǧ‹–‡‰‡”‹’—–•
and non-numeric inputs. So, we could generate the following invalid input equivalence partitions:

TCOND7: ‡šƒƒ”α”‡ƒŽ—„‡”™‹–Šƒˆ”ƒ…–‹‘ƒŽ’ƒ”– (for FS1)

TCOND8: ‡šƒƒ”αƒŽ’Šƒ„‡–‹… (for FS1)

TCOND9: ‡šƒƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for FS1)

TCOND10: …‘—”•‡™‘”ƒ”α”‡ƒŽ—„‡”™‹–Šƒˆ”ƒ…–‹‘ƒŽ’ƒ”– (for FS1)

TCOND11: …‘—”•‡™‘”ƒ”αƒŽ’Šƒ„‡–‹… (for FS1)

TCOND12: …‘—”•‡™‘”ƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for FS1)

44 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

‡š–ǡ–Š‡’ƒ”–‹–‹‘•ˆ‘”–Š‡˜ƒŽ‹†‘—–’—–•ƒ”‡‹†‡–‹ϐ‹‡†ǣ

TCOND13: Ǯǯ‹•‹†—…‡†„›͹Ͳδα–‘–ƒŽƒ”δαͳͲͲ (for FS1)

TCOND14: Ǯǯ‹•‹†—…‡†„›ͷͲδα–‘–ƒŽƒ”δ͹Ͳ (for FS1)

TCOND15: Ǯǯ‹•‹†—…‡†„›͵Ͳδα–‘–ƒŽƒ”δͷͲ (for FS1)

TCOND16: Ǯǯ‹•‹†—…‡†„›Ͳδα–‘–ƒŽƒ”δ͵Ͳ (for FS1)

TCOND17: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›–‘–ƒŽƒ”εͳͲͲ (for FS1)

TCOND18: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›–‘–ƒŽƒ”δͲ (for FS1)

TCOND19: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›‘Ǧ‹–‡‰‡”‹’—–• (for FS1)

™Š‡”‡–‘–ƒŽƒ”α‡šƒƒ”Ϊ…‘—”•‡™‘”ƒ”Ǥ‘–‡–Šƒ–Ǯ ƒ—Ž–‡••ƒ‰‡ǯ‹•…‘•‹†‡”‡†ƒ•ƒ˜ƒŽ‹†
output as it is a •’‡…‹ϔ‹‡† output.
Š‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ƒ†„‘—†ƒ”‹‡•ˆ‘”–‘–ƒŽƒ”ƒ”‡•Š‘™’‹…–‘”‹ƒŽŽ›„‡Ž‘™ǣ

Figure B.3 — Equivalence partitions and boundaries for total mark

‹˜ƒŽ‹†‘—–’—–™‘—Ž†„‡ƒ›‘—–’—–‘ˆ–Š‡–‡•–‹–‡‘–Š‡”–Šƒƒ›‘ˆ–Š‡ϐ‹˜‡•’‡…‹ϐ‹‡†Ǥ–…ƒ„‡
…ŠƒŽŽ‡‰‹‰–‘‹†‡–‹ˆ›—•’‡…‹ϐ‹‡†‘—–’—–•Ǥ‘™‡˜‡”ǡ–Š‡›—•–„‡…‘•‹†‡”‡†„‡…ƒ—•‡‹ˆ™‡…ƒ…ƒ—•‡
‘‡–‘‘……—”ǡ–Š‡™‡Šƒ˜‡‹†‡–‹ϐ‹‡†ƒϐŽƒ™™‹–Š–Š‡–‡•–‹–‡ǡ‹–•–‡•–„ƒ•‹•ǡ‘”„‘–ŠǤ ‘”–Š‹•‡šƒ’Ž‡
–Š”‡‡—•’‡…‹ϐ‹‡†‘—–’—–•™‡”‡‹†‡–‹ϐ‹‡†ƒ†ƒ”‡•Š‘™„‡Ž‘™ǤŠ‹•ƒ•’‡…–‘ˆ‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰
‹•˜‡”›•—„Œ‡…–‹˜‡ƒ††‹ˆˆ‡”‡––‡•–‡”•ƒ›‹†‡–‹ˆ›†‹ˆˆ‡”‡–’ƒ”–‹–‹‘•™Š‹…Štheyˆ‡‡Ž…‘—Ž†’‘••‹„Ž›
occur (this is described in 5.2.1.1 NOTE 2).

TCOND20: ‘—–’—–αǮǯǡ‹…ƒ•‡–Š‹•ȋ‹˜ƒŽ‹†Ȍ‡š–”ƒ‰”ƒ†‡Šƒ•„‡‡‹’Ž‡‡–‡† (for FS1)

in error for total marks from 0 to 15

TCOND21: ‘—–’—–αǮΪǯǡ‹…ƒ•‡–Š‹•ȋ‹˜ƒŽ‹†Ȍ‡š–”ƒ‰”ƒ†‡Šƒ•„‡‡‹’Ž‡‡–‡† (for FS1)

in error for total marks from 90 to 100

TCOND22: ‘—–’—–αǮ—ŽŽǯǡ‹…ƒ•‡ƒǮ—ŽŽǯ‘—–’—–…ƒ„‡‰‡‡”ƒ–‡† (for FS1)

B.2.1.5 Step 3: Derive Test Coverage Items (TD3)

In equivalence partitioning, the test coverage items are the partitions that were derived in the previous
step (i.e. test conditions are the same as test coverage items in this technique). Thus, the following test
…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡†‡ϐ‹‡†Ǥ

TCOVER1: Ͳδα‡šƒƒ”δα͹ͷ (for TCOND1)

TCOVER2: Ͳδα…‘—”•‡™‘”ƒ”δαʹͷ (for TCOND2)

© ISO/IEC 2015 – All rights reserved 45

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TCOVER3: ‡šƒƒ”δͲ (for TCOND3)

TCOVER4: exam mark > 75 (for TCOND4)

TCOVER5: …‘—”•‡™‘”ƒ”δͲ (for TCOND5)

TCOVER6: coursework mark > 25 (for TCOND6)

TCOVER7: ‡šƒƒ”α”‡ƒŽ—„‡”™‹–Šƒˆ”ƒ…–‹‘ƒŽ’ƒ”– (for TCOND7)

TCOVER8: ‡šƒƒ”αƒŽ’Šƒ„‡–‹… (for TCOND8)

TCOVER9: ‡šƒƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for TCOND9)

TCOVER10: …‘—”•‡™‘”ƒ”α”‡ƒŽ—„‡”™‹–Šƒˆ”ƒ…–‹‘ƒŽ’ƒ”– (for TCOND10)

TCOVER11: …‘—”•‡™‘”ƒ”αƒŽ’Šƒ„‡–‹… (for TCOND11)

TCOVER12: …‘—”•‡™‘”ƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for TCOND12)

TCOVER13: Ǯǯ‹•‹†—…‡†„›͹Ͳδα–‘–ƒŽƒ”δαͳͲͲ (for TCOND13)

TCOVER14: Ǯǯ‹•‹†—…‡†„›ͷͲδα–‘–ƒŽƒ”δ͹Ͳ (for TCOND14)

TCOVER15: Ǯǯ‹•‹†—…‡†„›͵Ͳδα–‘–ƒŽƒ”δͷͲ (for TCOND15)

TCOVER16: Ǯǯ‹•‹†—…‡†„›Ͳδα–‘–ƒŽƒ”δ͵Ͳ (for TCOND16)

TCOVER17: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›–‘–ƒŽƒ”εͳͲͲ (for TCOND17)

TCOVER18: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›–‘–ƒŽƒ”δͲ (for TCOND18)

TCOVER19: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ‹•‹†—…‡†„›‘Ǧ‹–‡‰‡”‹’—–ȋ•Ȍ (for TCOND19)

TCOVER20: ‘—–’—–αǮǯ (for TCOND20)

TCOVER21: ‘—–’—–αǮΪǯ (for TCOND21)

TCOVER22: ‘—–’—–αǮ—ŽŽǯ (for TCOND22)

B.2.1.6 Step 4: Derive Test Cases (TD4)

B.2.1.6.1 Options

ƒ˜‹‰‹†‡–‹ϐ‹‡†’ƒ”–‹–‹‘•ƒ†–‡•–…‘˜‡”ƒ‰‡‹–‡•–‘„‡–‡•–‡†ǡ–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†–Šƒ–ƒ––‡’–
to “hit” each test coverage item. Two common approaches for test case design are one-to-one and
minimized equivalence partitioning (other approaches to selecting combinations of test coverage
‹–‡•–‘„‡‡š‡”…‹•‡†„›–‡•–…ƒ•‡•–Šƒ–ƒ›„‡—•‡†ƒ”‡†‡•…”‹„‡†‹5.2.5ȌǤ–Š‡ϐ‹”•–ƒ–‡•–…ƒ•‡‹•
‰‡‡”ƒ–‡†ˆ‘”‡ƒ…Š‹†‡–‹ϐ‹‡†’ƒ”–‹–‹‘‘ƒ‘‡Ǧ–‘Ǧ‘‡„ƒ•‹•ȋ•‡‡‘’–‹‘Ͷƒ„‡Ž‘™Ȍǡ™Š‹Ž‡‹–Š‡•‡…‘†
ƒ‹‹‹œ‡†•‡–‘ˆ–‡•–…ƒ•‡•‹•‰‡‡”ƒ–‡†–‘…‘˜‡”ƒŽŽ–Š‡‹†‡–‹ϐ‹‡†’ƒ”–‹–‹‘•ȋ•‡‡‘’–‹‘Ͷ„„‡Ž‘™ȌǤ
The preconditions of all test cases for the generate_grading function are the same: that the application
‹•”‡ƒ†›–‘–ƒ‡–Š‡‹’—–•‘ˆ‡šƒƒ”ƒ†…‘—”•‡™‘”ƒ”Ǥ

B.2.1.6.2 Option 4a: Derive Test Cases for One-to-One Equivalence Partitioning (TD4)

Š‡ ‘‡Ǧ–‘Ǧ‘‡ ƒ’’”‘ƒ…Š ™‹ŽŽ „‡ †‡‘•–”ƒ–‡† ϐ‹”•– ƒ• ‹– …ƒ ƒ‡ ‹– ‡ƒ•‹‡” –‘ •‡‡ –Š‡ Ž‹ „‡–™‡‡
’ƒ”–‹–‹‘•ƒ†–‡•–…ƒ•‡•Ǥ ‘”‡ƒ…Š‘ˆ–Š‡•‡–‡•–…ƒ•‡•‘Ž›–Š‡•‹‰Ž‡–‡•–…‘˜‡”ƒ‰‡‹–‡„‡‹‰–ƒ”‰‡–‡†
‹••–ƒ–‡†‡š’Ž‹…‹–Ž›Ǥ™‡–›Ǧ–™‘–‡•–…‘˜‡”ƒ‰‡‹–‡•™‡”‡‹†‡–‹ϐ‹‡†Ž‡ƒ†‹‰–‘–™‡–›Ǧ–™‘–‡•–…ƒ•‡•Ǥ
The test cases corresponding to partitions derived from the input exam mark are shown below. Note
–Šƒ––Š‡‹’—–…‘—”•‡™‘”ƒ”‹–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…ƒ•‡–ƒ„Ž‡Šƒ•„‡‡•‡––‘ƒƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡
46 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

‘ˆͳͷǤŠ‡ƒŽŽ‘…ƒ–‹‘‘ˆƒƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡–‘ƒŽŽ‹’—–•‹–Š‡–‡•–…ƒ•‡ȋ‘–Š‡”–Šƒ–Š‡‘‡„‡‹‰
tested) has been carried out for all test cases in this clause.

Table B.1 — Test cases for input exam mark

Test Case 1 2 3
Input (exam mark) 60 -10 93
Input (c/w mark) 15 15 15
total mark (as calculated) 75 5 108
Test Coverage Item TCOVER1 TCOVER3 TCOVER4
Partition tested (of exam mark) Ͳδα‡δα͹ͷ ‡δͲ e > 75
Exp. Output Ǯǯ Ǯ ǯ Ǯ ǯ

The test cases corresponding to partitions derived from the input coursework mark are:

Table B.2 — Test cases for input coursework mark

Test Case 4 5 6
Input (exam mark) 40 40 40
Input (c/w mark) 20 -15 47
total mark (as calculated) 60 25 87
Test Coverage Item TCOVER2 TCOVER5 TCOVER6
Partition tested (of c/w mark) Ͳδα…δαʹͷ …δͲ c > 25
Exp. Output Ǯǯ Ǯ ǯ Ǯ ǯ

The test cases corresponding to partitions derived from possible invalid inputs are:

Table B.3 — Test cases for invalid inputs for exam mark

Test Case 7 8 9
Input (exam mark) 60.5 Q $
Input (c/w mark) 15 15 15
total mark (as calculated) 75.5 not applicable not applicable
Test Coverage Item TCOVER7 TCOVER8 TCOVER9
Partition tested ‡šƒƒ”α ‡šƒƒ”α ‡šƒƒ”α
real number with alphabetic special char
fractional part
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table B.4 — Test cases for invalid inputs for coursework mark

Test Case 10 11 12
Input (exam mark) 40 40 40
Input (c/w mark) 20.23 G ̷
total mark (as calculated) 60.23 not applicable not applicable
Test Coverage Item TCOVER10 TCOVER11 TCOVER12
Partition tested …Ȁ™ƒ”α …Ȁ™ƒ”αƒŽ’Šƒ- …Ȁ™ƒ”α•’‡…‹ƒŽ
real number with betic char
fractional part

© ISO/IEC 2015 – All rights reserved 47

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.4 (continued)

Test Case 10 11 12
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ

The test cases corresponding to partitions derived from the valid outputs are:

Table B.5 — Test cases for valid output total mark

Test Case 13 14 15
Input (exam mark) 60 44 32
Input (c/w mark) 20 22 13
total mark (as calculated) 80 66 45
Test Coverage Item TCOVER13 TCOVER14 TCOVER15
Partition tested (of total mark) ͹Ͳδα–δαͳͲͲ ͷͲδα–δ͹Ͳ ͵Ͳδα–δͷͲ
Exp. Output Ǯǯ Ǯǯ Ǯǯ

Table B.6 — Test cases for valid output total mark

Test Case 16 17 18
Input (exam mark) 12 80 -10
Input (c/w mark) 5 60 -10
total mark (as calculated) 17 140 -20
Test Coverage Item TCOVER16 TCOVER17 TCOVER18
Partition tested (of total mark) Ͳδα–δ͵Ͳ t > 100 –δͲ
Exp. Output Ǯǯ Ǯ ǯ Ǯ ǯ

The input values of exam mark and coursework mark have been derived from total mark, which is their
sum.
The test cases corresponding to partitions derived from the invalid outputs are:

Table B.7 — Test cases for invalid output total mark

Test Case 19 20 21 22
Input (exam mark) 47.3 5 72 Null
Input (c/w mark) ̷̷̷ 5 23 Null
total mark (as calculated) - 10 95 -
Test Coverage Item TCOVER19 TCOVER20 TCOVER21 TCOVER22
Partition tested (output) Ǯ ǯ Ǯǯ ǮΪǯ Ǯ—ŽŽǯ
Partition (of total mark) - Ͳδα–δαͳͷ ͻͲδα–δαͳͲͲ -
Exp. Output Ǯ ǯ Ǯǯ Ǯǯ Ǯ ǯ

‡’‡†‹‰‘–Š‡‹’Ž‡‡–ƒ–‹‘ǡ‹–ƒ›„‡‹’‘••‹„Ž‡–‘‡š‡…—–‡–‡•–…ƒ•‡•–Šƒ–…‘–ƒ‹‹˜ƒŽ‹†‹’—–
values (e.g. test cases 2, 3, 5-12, and 17-22 in the example above). For instance, in the Ada programming
language, if the input variable is declared as a positive integer then it will not be possible to assign a
negative value to it. Despite this, it is still worthwhile considering all the test cases for completeness.

B.2.1.6.3 Option 4b: Derive Test Cases for Minimized Equivalence Partitioning (TD4)

It can be seen above that several of the test cases are similar, such as test cases 1 and 13, where the
ƒ‹†‹ˆˆ‡”‡…‡„‡–™‡‡–Š‡‹•–Š‡•’‡…‹ϐ‹…–‡•–…‘˜‡”ƒ‰‡‹–‡…Š‘•‡ˆ”‘–Š‡’ƒ”–‹–‹‘–ƒ”‰‡–‡†Ǥ•
48 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

–Š‡–‡•–‹–‡Šƒ•–™‘‹’—–•ƒ†‘‡‘—–’—–ǡ‡ƒ…Š–‡•–…ƒ•‡ƒ…–—ƒŽŽ›DzŠ‹–•dz–Š”‡‡’ƒ”–‹–‹‘•Ǣ–™‘‹’—–
partitions and one output partition. Thus it is possible to generate a smaller “minimized” test set that
•–‹ŽŽDzŠ‹–•dzƒŽŽ–Š‡‹†‡–‹ϐ‹‡†’ƒ”–‹–‹‘•„›†‡”‹˜‹‰–‡•–…ƒ•‡•–Šƒ–ƒ”‡†‡•‹‰‡†–‘‡š‡”…‹•‡‘”‡–Šƒ
one partition.
The following test suite of twelve test cases corresponds to the minimized equivalence partitioning
ƒ’’”‘ƒ…Š™Š‡”‡‡ƒ…Š–‡•–…ƒ•‡‹•†‡•‹‰‡†–‘Š‹–ƒ•ƒ›‡™’ƒ”–‹–‹‘•ƒ•’‘••‹„Ž‡”ƒ–Š‡”–ŠƒŒ—•–‘‡Ǥ

Table B.8 — Minimized test cases

Test Case 1 2 3 4
Input (exam mark) 60 50 35 19
Input (c/w mark) 20 16 10 8
total mark (as calculated) 80 66 45 27
Test Coverage Items TCOVER1, TCOVER1, TCOVER1, TCOVER1,
TCOVER2, TCOVER2, TCOVER2, TCOVER2,
TCOVER13 TCOVER14 TCOVER15 TCOVER16
Partition (of exam mark) Ͳδα‡δα͹ͷ Ͳδα‡δα͹ͷ Ͳδα‡δα͹ͷ Ͳδα‡δα͹ͷ
Partition (of c/w mark) Ͳδα…δαʹͷ Ͳδα…δαʹͷ Ͳδα…δαʹͷ Ͳδα…δαʹͷ
Partition (of total mark) ͹Ͳδα–δαͳͲͲ ͷͲδα–δ͹Ͳ ͵Ͳδα–δͷͲ Ͳδα–δ͵Ͳ
Exp. Output Ǯǯ Ǯǯ Ǯǯ Ǯǯ

Table B.9 — Minimized test cases

Test Case 5 6 7 8
Input (exam mark) -10 93 60.5 Q
Input (c/w mark) -15 47 20.23 G
total mark (as calculated) -25 140 80.73 -
Test Coverage Items TCOVER3, TCOVER4, TCOVER7, TCOVER8,
TCOVER5, TCOVER6, TCOVER10, TCOVER11,
TCOVER18 TCOVER17 TCOVER13, TCOVER19
TCOVER19
Partition (of exam mark) ‡δͲ e > 75 ‡α”‡ƒŽ—„‡” ‡αƒŽ’Šƒ„‡–‹…
with fractional part
Partition (of c/w mark) …δͲ c > 25 …ᔇƒŽ—„‡” …αƒŽ’Šƒ„‡–‹…
with fractional part
Partition (of total mark) –δͲ t > 100 ͹Ͳδα–δαͳͲͲ -
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table B.10 — Minimized test cases

Test Case 9 10 11 12
Input (exam mark) $ 5 72 Ǯ—ŽŽǯ
Input (c/w mark) ̷ 5 23 Ǯ—ŽŽǯ
total mark (as calculated) - 10 95 -

© ISO/IEC 2015 – All rights reserved 49

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.10 (continued)

Test Case 9 10 11 12
Test Coverage Items TCOVER9, TCOVER1, TCOVER1, TCOVER19,
TCOVER12, TCOVER2, TCOVER2, TCOVER22
TCOVER19 TCOVER16, TCOVER13,
TCOVER20 TCOVER21
Partition (of exam mark) ‡α•’‡…‹ƒŽ…Šƒ” Ͳδα‡δα͹ͷ Ͳδα‡δα͹ͷ -
Partition (of c/w mark) …α•’‡…‹ƒŽ…Šƒ” Ͳδα…δαʹͷ Ͳδα…δαʹͷ -
Partition (of total mark) - Ͳδα–δαͳͷ ͻͲδα–δαͳͲͲ -
Partition (of output) - Ǯǯ ǮΪǯ Ǯ—ŽŽǯ
Exp. Output Ǯ ǯ Ǯǯ Ǯǯ Ǯ ǯ

The one-to-one and minimized approaches represent two different approaches that can be used for
†‡”‹˜‹‰–‡•–…ƒ•‡•ˆ‘”‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰Ǥ‡Ǧ–‘Ǧ‘‡–‡•–…ƒ•‡•ƒ”‡’ƒ”–‹…—Žƒ”Ž›—•‡ˆ—Žˆ‘”–‡•–‹‰
‡””‘”…‘†‹–‹‘•ȋ‹Ǥ‡Ǥ™Š‡›‘—ƒ”‡–”›‹‰–‘ˆ‘”…‡•’‡…‹ϐ‹…‡””‘”‡••ƒ‰‡•–‘„‡‘—–’—–Ȍǡˆ‘”‡šƒ’Ž‡ǡ
–‘”‡†—…‡–Š‡’‘••‹„‹Ž‹–›–Šƒ–‘‡‡””‘”…‘†‹–‹‘ŠƒŽ–•’”‘…‡••‹‰ƒ†Ȁ‘”ƒ••‘”„Ž‘…•‘–Š‡”‡””‘”
conditions. On the other hand, the disadvantage of the one-to-one approach is that it requires more test
cases and if this causes problems, a more minimalist approach can be used. The disadvantage of the
‹‹ƒŽ‹•–ƒ’’”‘ƒ…Š‹•–Šƒ–‹–Š‡‡˜‡–‘ˆƒ–‡•–ˆƒ‹Ž—”‡‹–…ƒ„‡†‹ˆϐ‹…—Ž––‘‹†‡–‹ˆ›–Š‡…ƒ—•‡†—‡–‘
several new partitions being exercised at the same time. Therefore, a common approach is to combine
–Š‡–™‘ƒ’’”‘ƒ…Š‡•„›ƒ’’Ž›‹‰‹‹‹œ‡†‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰–‘†‡•‹‰˜ƒŽ‹†–‡•–…ƒ•‡•ƒ†‘‡Ǧ
to-one equivalence partitioning to design invalid test cases.

B.2.1.7 Step 5: Assemble Test Sets (TD5)

B.2.1.7.1 Options

ˆ™‡ƒ••—‡–Šƒ–‹–‹•’‘••‹„Ž‡–‘ƒ—–‘ƒ–‹…ƒŽŽ›…Š‡…ƒƒ……‡’–Ȁ”‡Œ‡…–”‡•’‘•‡ˆ‘”‡ƒ…Š–‡•–…ƒ•‡ǡ„—–
–Šƒ–ƒ—–‘ƒ–‹‘…ƒ‘–Šƒ†Ž‡ˆƒ—Ž–‡••ƒ‰‡•ȋ Ȍǡ–Š‡™‡…ƒ‰‡‡”ƒ–‡–™‘–‡•–•‡–•ȋȌǢ‘‡ˆ‘”
manual testing and one for automated testing.

B.2.1.7.2 Option 5a: Assemble Test Set for One-to-One Equivalence Partitioning (TD5)

ͳǣƒ—ƒŽ‡•–‹‰Ȃʹǡ͵ǡͷǡ͸ǡ͹ǡͺǡͻǡͳͲǡͳͳǡͳʹǡͳ͹ǡͳͺǡͳͻǡʹʹǤ

TS2: Automated Testing – TEST CASES 1, 4, 13, 14, 15, 16, 20, 21.

B.2.1.7.3 Option 5b: Assemble Test Set for Minimized Equivalence Partitioning (TD5)

͵ǣƒ—ƒŽ‡•–‹‰Ȃͷǡ͸ǡ͹ǡͺǡͻǡͳʹǤ

TS4: Automated Testing – TEST CASES 1, 2, 3, 4, 10, 11.

B.2.1.8 Step 6: Derive Test Procedures (TD6)

B.2.1.8.1 Options

Test procedures for one-to-one and minimized equivalence partitioning can now be derived.

B.2.1.8.2 Option 6a: Derive Test Procedures for One-to-One Equivalence Partitioning (TD6)

For the manual test cases in test set TS1 for one-to-one equivalence partitioning, one test procedure
ȋȌ…ƒ„‡†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•ǣ

50 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

ͳǣƒ—ƒŽ–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ
For the automated test cases in one-to-one test set TS2, one test script could be written to execute all
test cases in the test set, as follows:
ʹǣƒ—–‘ƒ–‡†–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ
For the automated test procedures TP2, automation code that implements the procedure would need to
be written in test automation scripts.

B.2.1.8.3 Option 6b: Derive Test Procedures for Minimized Equivalence Partitioning (TD6)

‘”–Š‡ƒ—ƒŽ–‡•–…ƒ•‡•‹‹‹‹œ‡†–‡•–•‡–͵ǡ‘‡–‡•–’”‘…‡†—”‡ȋȌ…ƒ„‡†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•ǣ
͵ǣƒ—ƒŽ–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹͵ǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ
For the automated test cases in minimized test set TS4, one test script could be written to execute all
test cases in the test set, as follows:
Ͷǣƒ—–‘ƒ–‡†–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹Ͷǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.1.9 Equivalence Partition Coverage

Using the formula provided in 6.2.1 and the test coverage items derived above:
22
Coverage(one −to−one _ EP ) = × 100% = 100%
22
12
Coverage(minimized_EP ) = × 100% = 100%
12
Thus, 100% equivalence partition coverage was achieved for both one-to-one and minimized
‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰ǡ‡ƒ„Ž‹‰ƒŽŽ–™‡–›Ǧ–™‘‹†‡–‹ϐ‹‡†’ƒ”–‹–‹‘•–‘„‡‡š‡”…‹•‡†„›ƒ–Ž‡ƒ•–‘‡
–‡•–…ƒ•‡Ǥ‘™‡”…‘˜‡”ƒ‰‡Ž‡˜‡Ž•™‘—Ž†„‡ƒ…Š‹‡˜‡†‹ˆ‘–ƒŽŽ’ƒ”–‹–‹‘•‹†‡–‹ϐ‹‡†ƒ”‡‡š‡”…‹•‡†Ǥˆ‘–
ƒŽŽ’ƒ”–‹–‹‘•ƒ”‡‹†‡–‹ϐ‹‡†ǡ–Š‡ƒ›…‘˜‡”ƒ‰‡‡ƒ•—”‡„ƒ•‡†‘–Š‹•‹…‘’Ž‡–‡•‡–‘ˆ’ƒ”–‹–‹‘•…‘—Ž†
„‡‹•Ž‡ƒ†‹‰Ǥ‘™‡˜‡”ǡ•‹…‡ƒ†‹ˆˆ‡”‡–ƒƒŽ›•‹•‘ˆ–Š‡–‡•–‹–‡‹‰Š–‹†‡–‹ˆ›†‹ˆˆ‡”‡–‡“—‹˜ƒŽ‡…‡
’ƒ”–‹–‹‘•ǡ ’ƒ”–‹…—Žƒ”Ž› ˆ‘” Dz‹˜ƒŽ‹†dz ˜ƒŽ—‡•ǡ …‘˜‡”ƒ‰‡ ‡ƒ•—”‡• ˆ‘” ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘‹‰ —•– „‡
“—ƒŽ‹ϐ‹‡†ƒ•„‡‹‰ˆ‘”Dz‹†‡–‹ϐ‹‡†dz’ƒ”–‹–‹‘•Ǥ

ǤʹǤʹ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†

B.2.2.1 Introduction

Š‡ƒ‹‘ˆ–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†‹•–‘†‡”‹˜‡–‡•–…ƒ•‡•–Šƒ–…‘˜‡”–Š‡‹’—–’ƒ”–‹–‹‘•‘ˆ–Š‡
–‡•–‹–‡ƒ……‘”†‹‰–‘–Š‡…Š‘•‡Ž‡˜‡Ž‘ˆ‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘…‘˜‡”ƒ‰‡ǤŠ‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†
‡š–‡†•–Š‹•…‘…‡’–„›…‘•–”—…–‹‰ƒ…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡–Šƒ–‹ŽŽ—•–”ƒ–‡•–Š‡’ƒ”–‹–‹‘•ƒ†ƒ••‹•–•–Š‡
tester with test design.

ǤʹǤʹǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the test basis for a test item travel_preference, which records the travel preferences of staff of an
—•–”ƒŽ‹ƒ‘”‰ƒ‹•ƒ–‹‘™Š‘–”ƒ˜‡Ž–‘ƒŒ‘”—•–”ƒŽ‹ƒ…ƒ’‹–ƒŽ…‹–‹‡•ˆ‘”™‘”’—”’‘•‡•Ǥƒ…Š•‡–‘ˆ–”ƒ˜‡Ž
preferences is chosen through a series of radio buttons, which consist of the following input value choices:
Destination = Adelaide, Brisbane, Canberra, Darwin, Hobart, Melbourne, Perth, Sydney
Class = First Class, Business Class, Economy
Seat = Aisle, Window

© ISO/IEC 2015 – All rights reserved 51

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Meal Preference = diabetic, gluten free, lacto-ovo vegetarian, low fat/cholesterol, low lactose, vegan
vegetarian, standard
›…‘„‹ƒ–‹‘‘ˆ‘‡…Žƒ••ˆ”‘‡ƒ…Š…Žƒ••‹ϔ‹…ƒ–‹‘™‹ŽŽ”‡•—Ž–‹–Š‡‡••ƒ‰‡Dz‘‘‹‰ƒ……‡’–‡†dz™Š‹Ž‡
ƒ› ‘–Š‡” ‹’—– ™‹ŽŽ ”‡•—Ž– ‹ ƒ ‡””‘” ‡••ƒ‰‡ •–ƒ–‹‰ Dz‹˜ƒŽ‹† ‹’—–dzǤ –ƒˆˆ †‘ ‘– Šƒ˜‡ –Š‡ ‘’–‹‘ ‘ˆ
choosing no meal, thus this option is not supported in the example.

B.2.2.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
ͳǣ–”ƒ˜‡Ž̴’”‡ˆ‡”‡…‡ˆ—…–‹‘

B.2.2.4 Step 2: Derive Test Conditions (TD2)

‘”…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡–‡•–‹‰ǡ–‡•–…‘†‹–‹‘•ƒ”‡‹†‡–‹ϐ‹‡†„›†‡”‹˜‹‰…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ†…Žƒ••‡•ˆ‘”
each input parameter.

TCOND1: Destination (for FS1)

TCOND2: Class (for FS1)

TCOND3: Seat (for FS1)

TCOND4: ‡ƒŽ”‡ˆ‡”‡…‡ (for FS1)

NOTE 1 Invalid test conditions could also be derived, though these are not demonstrated in this example.

ƒ…Š ‘ˆ –Š‡ –‡•– …‘†‹–‹‘• ‹ –Š‹• ‡šƒ’Ž‡ ‹• ƒ Dz…Žƒ••‹ϐ‹…ƒ–‹‘dz ȋ‹Ǥ‡Ǥ ƒ ’ƒ”–‹–‹‘ȌǤ DzŽƒ••‡•dz ȋ‹Ǥ‡Ǥ •—„Ǧ
’ƒ”–‹–‹‘•Ȍƒ†•—„Ǧ…Žƒ••‡•…‘—Ž†„‡†‡”‹˜‡†ˆ‘”–Š‡‡ƒŽ”‡ˆ‡”‡…‡…Žƒ••ƒ•ˆ‘ŽŽ‘™•ǣ

‡ƒŽ”‡ˆ‡”‡…‡ȋ…Žƒ••‹ϐ‹…ƒ–‹‘Ȍᇉ‡–ƒ”‹ƒǡ‘Ǧ˜‡‰‡–ƒ”‹ƒ

‡‰‡–ƒ”‹ƒȋ…Žƒ••ȌαŽƒ…–‘Ǧ‘˜‘ǡ˜‡‰ƒ

‘Ǧ˜‡‰‡–ƒ”‹ƒȋ…Žƒ••Ȍᆋƒ„‡–‹…ǡ‰Ž—–‡ˆ”‡‡ǡŽ‘™ˆƒ–Ȁ…Š‘Ž‡•–‡”‘ŽǡŽ‘™Žƒ…–‘•‡ǡ•–ƒ†ƒ”†
ʹ Š‡ †‡•‹‰ ‘ˆ …Žƒ••‹ϐ‹…ƒ–‹‘• ƒ† …Žƒ••‡• ‹• ‘ˆ–‡ ƒ •—„Œ‡…–‹˜‡ ƒ…–‹˜‹–›ǡ –Š—• ‘–Š‡” –‡•–‡”• —•‹‰ –Š‹•
–‡…Š‹“—‡…ƒ†‡•‹‰…Žƒ••‹ϐ‹…ƒ–‹‘•ƒ†…Žƒ••‡•–Šƒ–†‹ˆˆ‡”ˆ”‘–Š‘•‡†‡”‹˜‡†‹–Š‹•‡šƒ’Ž‡Ǥ

…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡…ƒ‘™„‡†‡˜‡Ž‘’‡†ˆ‘”–Š‡•‡–‡•–…‘†‹–‹‘•Ǥ

‹‰—”‡ǤͶȄŽƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡šƒ’Ž‡

B.2.2.5 Step 3: Derive Test Coverage Items (TD3)

‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡†‡”‹˜‡†„›…Š‘‘•‹‰ƒ…‘„‹ƒ–‹‘ƒ’’”‘ƒ…Šƒ†–Š‡…”‡ƒ–‹‰…‘„‹ƒ–‹‘•
‘ˆ…Žƒ••‡•ƒ……‘”†‹‰–‘–Šƒ–ƒ’’”‘ƒ…ŠǤDz…‘„‹ƒ–‹‘–ƒ„Ž‡dz…ƒ„‡…‘•–”—…–‡†—†‡”–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘
52 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

tree to demonstrate which classes are combined to form each test coverage item (e.g. see Figure B.5
„‡Ž‘™ȌǤŠ‡…Žƒ••‡•–Šƒ–ƒ”‡…‘˜‡”‡†„›‡ƒ…Š–‡•–…‘˜‡”ƒ‰‡‹–‡ƒ”‡ƒ”‡†„›ƒ•‡”‹‡•‘ˆ–‘‡•ȋ„Žƒ…
†‘–•Ȍ–Šƒ–”—Š‘”‹œ‘–ƒŽŽ›—†‡”‡ƒ–Š–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡Ǥ
If we assume that the chosen combination approach is “minimized”, in which each test coverage item
…‘˜‡”• ƒ• ƒ› …Žƒ••‡• ƒ• ’‘••‹„Ž‡ —–‹Ž ƒŽŽ …Žƒ••‡• ƒ”‡ ‹…Ž—†‡† ‹ ƒ– Ž‡ƒ•– ‘‡ –‡•– …ƒ•‡ǡ –Š‡ –Š‡
ˆ‘ŽŽ‘™‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡••Š‘™‹ ‹‰—”‡Ǥͷ„‡Ž‘™…‘—Ž†„‡†‡ϐ‹‡†Ǥ
travel_preference
Classiication Tree

Destination Class Seat Meal Preference

Adelaide Brisbane Canberra Darwin Hobart Melbourne Perth Sydney First Business Economy Aisle Window Vegetarian Non-vegetarian

lacto- vegan diabetic gluten low fat / low standard

ovo free cholesterol lactose

1
2
Test Coverage Items

3
4
5
6
7
8

‹‰—”‡ǤͷȄšƒ’Ž‡‘ˆƒ…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡ƒ†…‘””‡•’‘†‹‰…‘„‹ƒ–‹‘–ƒ„Ž‡

In this example, all test coverage items cover all test conditions.

B.2.2.6 Step 4: Derive Test Cases (TD4)

•‡–‘ˆ–‡•–…ƒ•‡•…ƒ‘™„‡†‡”‹˜‡†ǡ™Š‡”‡‡ƒ…Š–‡•–…ƒ•‡…‘˜‡”•‡šƒ…–Ž›‘‡–‡•–…‘˜‡”ƒ‰‡‹–‡Ǥ‡•–
…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰‘‡–‡•–…‘˜‡”ƒ‰‡‹–‡ƒ–ƒ–‹‡–Šƒ–Šƒ•‘–ƒŽ”‡ƒ†›„‡‡…‘˜‡”‡†„›
a test case and populating it with test input values that cover the classes of that combination. This is
”‡’‡ƒ–‡† —–‹Ž –Š‡ ”‡“—‹”‡† Ž‡˜‡Ž ‘ˆ …‘˜‡”ƒ‰‡ ‹• ƒ…Š‹‡˜‡†Ǥ Š‡‡š’‡…–‡† ”‡•—Ž– ‹• †‡”‹˜‡† „› ƒ’’Ž›‹‰
–Š‡‹’—–•–‘–Š‡–‡•–„ƒ•‹•Ǥ–Š‹•’ƒ”–‹…—Žƒ”…ƒ•‡ǡƒ›…‘„‹ƒ–‹‘‘ˆ˜ƒŽ‹†‹’—–•”‡•—Ž–•‹–Š‡•–ƒ–—•
“Booking accepted”.

ƒ„Ž‡ǤͳͳȄ‡•–…ƒ•‡•ˆ‘”…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡–‡•–‹‰

Input Values
Test Expected Result Test Coverage
Case Destination Class Seat ‡ƒŽ”‡ˆ‡”‡…‡ Item
1 Adelaide First Aisle lacto-ovo Booking accepted TCOVER1
2 Brisbane Business Window vegan Booking accepted TCOVER2
3 Canberra …‘‘› Aisle diabetic Booking accepted TCOVER3
4 Darwin First Window gluten free Booking accepted TCOVER4
5 Hobart Business Aisle low fat/cholesterol Booking accepted TCOVER5
6 ‡Ž„‘—”‡ …‘‘› Window low lactose Booking accepted TCOVER6

© ISO/IEC 2015 – All rights reserved 53

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.11 (continued)

Input Values
Test Expected Result Test Coverage
Case Destination Class Seat ‡ƒŽ”‡ˆ‡”‡…‡ Item
7 Perth First Aisle standard Booking accepted TCOVER7
8 ›†‡› Business Window lacto-ovo Booking accepted TCOVER8

B.2.2.7 Step 5: Assemble Test Sets (TD5)

‹…‡ ƒ •ƒŽŽ —„‡” ‘ˆ –‡•– …ƒ•‡• ™‡”‡ †‡”‹˜‡† ‹ –Š‹• ‡šƒ’Ž‡ǡ ‹– ƒ› „‡ †‡…‹†‡† –Šƒ– –Š‡› …ƒ „‡
combined into the one test set.
TS1: TEST CASES 1, 2, 3, 4, 5, 6, 7, 8

B.2.2.8 Step 6: Derive Test Procedures (TD6)

Since all test cases are in the one test set, we can derive one test procedure.
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

ǤʹǤʹǤͻ Žƒ••‹ϐ‹…ƒ–‹‘”‡‡‡–Š‘†‘˜‡”ƒ‰‡

Using the formula provided in 6.2.2 and the test coverage items derived above:
8
Coverage(classification _ tree _ method ) = × 100% = 100%
8
Š—•ǡͳͲͲΨ…‘˜‡”ƒ‰‡‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”–Š‡…Žƒ••‹ϐ‹…ƒ–‹‘–”‡‡‡–Š‘†Šƒ•„‡‡ƒ…Š‹‡˜‡†Ǥ

B.2.3 Boundary Value Analysis

B.2.3.1 Introduction

Š‡ƒ‹‘ˆ„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•‹•–‘†‡”‹˜‡ƒ•‡–‘ˆ–‡•–…ƒ•‡•–Šƒ–…‘˜‡”–Š‡„‘—†ƒ”‹‡•‘ˆ‡ƒ…Š
‹’—–ƒ†‘—–’—–’ƒ”–‹–‹‘‘ˆ–Š‡–‡•–‹–‡ƒ……‘”†‹‰–‘–Š‡…Š‘•‡Ž‡˜‡Ž‘ˆ„‘—†ƒ”›˜ƒŽ—‡…‘˜‡”ƒ‰‡Ǥ–
is based on the following premises. First, that the inputs and outputs of a test item can be partitioned
‹–‘ …Žƒ••‡• –Šƒ–ǡ ƒ……‘”†‹‰ –‘ –Š‡ –‡•– „ƒ•‹• ˆ‘” –Š‡ –‡•– ‹–‡ǡ ™‹ŽŽ „‡ –”‡ƒ–‡† •‹‹Žƒ”Ž› „› –Š‡ –‡•–
item; second, that the members of some partitions can be ordered from lowest to highest with no
†‹•…‘–‹—‹–›Ǣƒ†–Š‹”†ǡ–Šƒ––Š‡„‘—†ƒ”‹‡•‘ˆ‘”†‡”‡†…‘–‹‰—‘—•’ƒ”–‹–‹‘•ƒ”‡Š‹•–‘”‹…ƒŽŽ›ƒ‡””‘”
prone element of software development. Test cases are generated to exercise these boundaries.
The following is an example of three-value boundary testing with one-to-one test cases (see 5.2.3.2
and 5.2.3.3”‡•’‡…–‹˜‡Ž›ȌǤ‘”†‡”–‘†‡”‹˜‡„‘—†ƒ”‹‡•ˆ‘”ƒ–‡•–‹–‡ǡ–Š‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•‘ˆ
–Š‡ –‡•– ‹–‡ —•– „‡ ‹†‡–‹ϐ‹‡† ϐ‹”•–ǡ ˆ‘ŽŽ‘™‡† „› –Š‡ †‡”‹˜ƒ–‹‘ ‘ˆ –Š‡ „‘—†ƒ”› ˜ƒŽ—‡• ˆ”‘ ‡ƒ…Š
equivalence class.

ǤʹǤ͵Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider a test item, generate_grading, with the following test basis:

The component receives an exam mark (out of 75) and a coursework (c/w) mark (out of 25) as input, from
which it outputs a grade for the course in the range ‘A’ to ‘D’. The grade is generated by calculating the
overall mark, which is the sum of the exam and c/w marks, as follows:

greater than or equal to 70 - ‘A’

greater than or equal to 50, but less than 70 - ‘B’

54 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

greater than or equal to 30, but less than 50 - ‘C’

less than 30 - ‘D’

Where invalid input(s) are detected (e.g. a mark is outside its expected range) then a fault message (‘FM’) is
generated. All inputs are passed as integers.

B.2.3.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–ȋ Ȍ‡‡†•–‘„‡†‡ϐ‹‡†ǣ
ͳǣ‰‡‡”ƒ–‡̴‰”ƒ†‹‰ˆ—…–‹‘

B.2.3.4 Step 2: Derive Test Conditions (TD2)

B.2.3.4.1 Sub-steps

‘”„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ǡ–Š‡–‡•–…‘†‹–‹‘•ƒ”‡–Š‡„‘—†ƒ”‹‡•ȋ„‡–™‡‡’ƒ”–‹–‹‘•Ȍ–Šƒ–Šƒ˜‡
„‡‡ …Š‘•‡ –‘ „‡ …‘˜‡”‡† †—”‹‰ –‡•–‹‰Ǥ ‘ ‹†‡–‹ˆ› –Š‡ „‘—†ƒ”‹‡•ǡ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘• —•–
ϐ‹”•–„‡‹†‡–‹ϐ‹‡†ȋ•‡‡•–‡’ʹƒ„‡Ž‘™Ȍǡˆ”‘™Š‹…Š–‡•–…‘†‹–‹‘•ȋ–Š‡„‘—†ƒ”‹‡•Ȍ…ƒ„‡†‡”‹˜‡†ȋ•‡‡
step 2b below).

B.2.3.4.2 Step 2a: Identify Equivalence Partitions

“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ƒ”‡‹†‡–‹ϐ‹‡†ˆ”‘–Š‡˜ƒŽ‹†ƒ†‹˜ƒŽ‹†‹’—–•ƒ†‘—–’—–•‘ˆ–Š‡ˆ‡ƒ–—”‡•‡– ͳǤ
The following valid‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ȋȌ…ƒ„‡‹†‡–‹ϐ‹‡†ˆ‘”–Š‡‹’—–•ǣ

EP1: Ͳδα‡šƒƒ”δα͹ͷ (for FS1)

EP2: Ͳδα…‘—”•‡™‘”ƒ”δαʹͷ (for FS1)

The most obvious invalid‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ˆ‘”–Š‡‹’—–•…ƒ„‡‹†‡–‹ϐ‹‡†ƒ•ǣ

EP3: exam mark > 75 (for FS1)

EP4: ‡šƒƒ”δͲ (for FS1)

EP5: coursework mark > 25 (for FS1)

EP6: …‘—”•‡™‘”ƒ”δͲ (for FS1)

Ž–Š‘—‰Š ’ƒ”–‹–‹‘• ͵ –‘ ͸ ƒ’’‡ƒ” –‘ „‡ „‘—†‡† ‘ ‘‡ •‹†‡ ‘Ž›ǡ –Š‡•‡ ’ƒ”–‹–‹‘• ƒ”‡ ‹ ˆƒ…–
„‘—†‡†„›‹’Ž‡‡–ƒ–‹‘Ǧ†‡’‡†‡–‹‹—ƒ†ƒš‹—˜ƒŽ—‡•Ǥ ‘”‹–‡‰‡”•Š‡Ž†‹•‹š–‡‡„‹–•
–Š‡•‡™‘—Ž†„‡͵ʹ͹͸͹ƒ†Ǧ͵ʹ͹͸ͺ”‡•’‡…–‹˜‡Ž›ǤŠ‡”‡ˆ‘”‡ǡ͵–‘͸…ƒ„‡‘”‡ˆ—ŽŽ›†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•ǣ

EP3: ͹ͷδ‡šƒƒ”δα͵ʹ͹͸͹ (for FS1)

EP4: Ǧ͵ʹ͹͸ͺδα‡šƒƒ”δͲ (for FS1)

EP5: ʹͷδ…‘—”•‡™‘”ƒ”δα͵ʹ͹͸͹ (for FS1)

EP6: Ǧ͵ʹ͹͸ͺδα…‘—”•‡™‘”ƒ”δͲ (for FS1)

ƒ”–‹–‹‘‡†”ƒ‰‡•‘ˆ˜ƒŽ—‡•…ƒ„‡”‡’”‡•‡–‡†’‹…–‘”‹ƒŽŽ›ƒ•ˆ‘ŽŽ‘™•ǣ

© ISO/IEC 2015 – All rights reserved 55

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Figure B.6 — Equivalence partitions and boundaries of exam mark

And for the input, coursework mark, we get:

Figure B.7 — Equivalence partitions and boundaries of coursework mark

‡•• ‘„˜‹‘—• ‹˜ƒŽ‹† ‹’—– ’ƒ”–‹–‹‘• …‘—Ž† ‹…Ž—†‡ ƒ› ‘–Š‡” ‹’—– –›’‡• ‘– ‹…Ž—†‡† ‹ ƒ ˜ƒŽ‹†
partition, for instance, non-integer inputs or perhaps non-numeric inputs. This aspect of equivalence
’ƒ”–‹–‹‘‹‰…ƒ„‡•—„Œ‡…–‹˜‡ƒ†ƒ••—…Šǡ‡ƒ…Š–‡•–‡”ƒ›‹†‡–‹ˆ›†‹ˆˆ‡”‡–’ƒ”–‹–‹‘•–Šƒ––Š‡›ˆ‡‡Ž
could be relevant. In order to be considered an equivalence partition, all values within a partition
—•–„‡‡š’‡…–‡†–‘„‡–”‡ƒ–‡†‹ƒ‡“—‹˜ƒŽ‡–ƒ‡”„›–Š‡–‡•–‹–‡ǤŠ—•™‡…‘—Ž†‰‡‡”ƒ–‡–Š‡
ˆ‘ŽŽ‘™‹‰‹˜ƒŽ‹†‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ˆ‘”–Š‡–™‘‹’—–ϐ‹‡Ž†•ǣ

EP7: ‡šƒƒ”α”‡ƒŽ—„‡”™‹–Šˆ”ƒ…–‹‘ƒŽ’ƒ”– (for FS1)

EP8: ‡šƒƒ”αƒŽ’Šƒ„‡–‹… (for FS1)

EP9: ‡šƒƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for FS1)

EP10: …‘—”•‡™‘”ƒ”α”‡ƒŽ—„‡”™‹–Šˆ”ƒ…–‹‘ƒŽ’ƒ”– (for FS1)

EP11: …‘—”•‡™‘”ƒ”αƒŽ’Šƒ„‡–‹… (for FS1)

EP12: …‘—”•‡™‘”ƒ”α•’‡…‹ƒŽ…Šƒ”ƒ…–‡” (for FS1)

Ž–Š‘—‰Š‡“—‹˜ƒŽ‡…‡…Žƒ••‡•͹–‘ͳʹƒ”‡’‘••‹„Ž‡ǡ–Š‡›Šƒ˜‡‘‹†‡–‹ϐ‹ƒ„Ž‡„‘—†ƒ”‹‡•ƒ†•‘‘
test coverage items or test cases need to be derived for them for this technique.
‡š–ǡ –Š‡ ’ƒ”–‹–‹‘• ˆ‘” –Š‡ ‘—–’—–• ƒ”‡ ‹†‡–‹ϐ‹‡†Ǥ Š‡ valid ’ƒ”–‹–‹‘• ƒ”‡ ’”‘†—…‡† „› …‘•‹†‡”‹‰
each of the valid outputs for the test item thus:

EP13: Ǯǯ‹•‹†—…‡†„›͹Ͳδα–‘–ƒŽƒ”δαͳͲͲ (for FS1)

EP14: Ǯǯ‹•‹†—…‡†„›ͷͲδα–‘–ƒŽƒ”δ͹Ͳ (for FS1)

EP15: Ǯǯ‹•‹†—…‡†„›͵Ͳδα–‘–ƒŽƒ”δͷͲ (for FS1)

EP16: Ǯǯ‹•‹†—…‡†„›Ͳδα–‘–ƒŽƒ”δ͵Ͳ (for FS1)

EP17: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ ‹•‹†—…‡†„›–‘–ƒŽƒ”εͳͲͲ (for FS1)

EP18: Ǯ ƒ—Ž–‡••ƒ‰‡ǯȋ Ȍ ‹•‹†—…‡†„›–‘–ƒŽƒ”δͲ (for FS1)

™Š‡”‡–‘–ƒŽƒ”α‡šƒƒ”Ϊ…‘—”•‡™‘”ƒ”Ǥ

56 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

‹‹Žƒ”–‘–Š‡‹’—–•ǡ–Š‡‘—–’—–‹•„‘—†‡†‘‡‹–Š‡”•‹†‡„›‹’Ž‡‡–ƒ–‹‘Ǧ†‡’‡†‡–ƒš‹—ƒ†
minimum values. Assuming the output is stored in integers held in sixteen bits from -32768 to 32767,
ͳ͹ƒ†ͳͺ…‘—Ž†„‡”‡†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•Ǥ

EP17: ͳͲͲδ–‘–ƒŽƒ”δα͵ʹ͹͸͹ (for FS1)

EP18: Ǧ͵ʹ͹͸ͺδα–‘–ƒŽƒ”δͲ (for FS1)

Ǯ ƒ—Ž–‡••ƒ‰‡ǯ‹•…‘•‹†‡”‡†Š‡”‡ƒ•‹–‹•ƒ•’‡…‹ϐ‹‡†‘—–’—–ǤŠ‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ƒ†„‘—†ƒ”‹‡•
for total mark are shown in Figure B.8:

Figure B.8 — Equivalence partitions and boundaries of total mark

An invalid ‘—–’—– ™‘—Ž† „‡ ƒ› ‘—–’—– ˆ”‘ –Š‡ –‡•– ‹–‡ ‘–Š‡” –Šƒ ‘‡ ‘ˆ –Š‡ ϐ‹˜‡ •’‡…‹ϐ‹‡†Ǥ – ƒ›
„‡ †‹ˆϐ‹…—Ž– –‘ ‹†‡–‹ˆ› —•’‡…‹ϐ‹‡† ‘—–’—–•ǡ „—– ‘„˜‹‘—•Ž› –Š‡› —•– „‡ …‘•‹†‡”‡† ƒ• ‹ˆ ™‡ …ƒ …ƒ—•‡
‘‡–Š‡™‡Šƒ˜‡‹†‡–‹ϐ‹‡†ƒ†‡ˆ‡…–™‹–Š–Š‡–‡•–‹–‡ǡ‹–•–‡•–„ƒ•‹•ǡ‘”„‘–ŠǤ ‘”–Š‹•‡šƒ’Ž‡–Š”‡‡
—•’‡…‹ϐ‹‡† ‘—–’—–• ™‡”‡ ‹†‡–‹ϐ‹‡† ȋǮǯǡ ǮΪǯǡ ƒ† Ǯ—ŽŽǯȌǡ „—– ‹– ‹• ‘– ’‘••‹„Ž‡ –‘ ‰”‘—’ –Š‡•‡ ’‘••‹„Ž‡
‘—–’—–•‹–‘‘”†‡”‡†’ƒ”–‹–‹‘•ˆ”‘™Š‹…Š„‘—†ƒ”‹‡•…ƒ„‡‹†‡–‹ϐ‹‡†ƒ†•‘‘–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†Ǥ

B.2.3.4.3 Step 2b: Derive Test Conditions

…‡ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘•ˆ‘”‡ƒ…Š ‹’—– ƒ†‘—–’—– ϐ‹‡Ž†Šƒ˜‡ „‡‡ ‹†‡–‹ϐ‹‡†ǡ –Š‡ –‡•– …‘†‹–‹‘•ǡ
™Š‹…Šƒ”‡–Š‡„‘—†ƒ”‹‡•‘ˆ‡ƒ…Š‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘ǡ…ƒ‘™„‡‹†‡–‹ϐ‹‡†Ǥ
For the valid’ƒ”–‹–‹‘•‘ˆ–Š‡‹’—–ϐ‹‡Ž†•‡šƒƒ”ƒ†…‘—”•‡™‘”ƒ”ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘†‹–‹‘•
…ƒ„‡‹†‡–‹ϐ‹‡†Ǥ‘–‡–Šƒ–†—’Ž‹…ƒ–‡„‘—†ƒ”‹‡•ȋ‡Ǥ‰Ǥ–Š‡„‘—†ƒ”›DzͲdz–Šƒ–Ž‹‡•‘–Š‡‡†‰‡•‘ˆͳƒ†
ͶȌƒ”‡‘Ž›…‘˜‡”‡†„›‘‡–‡•–…‘†‹–‹‘Ǥ

TCOND1: ‡šƒƒ”αͲ (for EP1 and EP4)

TCOND2: ‡šƒƒ”α͹ͷ (for EP1 and EP3)

TCOND3: …‘—”•‡™‘”ƒ”αͲ (for EP2 and EP6)

TCOND4: …‘—”•‡™‘”ƒ”αʹͷ (for EP2 and EP5)

For the valid‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘•ˆ‘”–‘–ƒŽƒ”ǡ–Š‡ˆ‘ŽŽ‘™‹‰„‘—†ƒ”‹‡•…ƒ„‡‹†‡–‹ϐ‹‡†Ǥ

TCOND5: –‘–ƒŽƒ”αͲ (for EP16 and EP18)

TCOND6: –‘–ƒŽƒ”αʹͻ (for EP15 and EP16)

TCOND7: –‘–ƒŽƒ”α͵Ͳ (for EP15 and EP16)

TCOND8: –‘–ƒŽƒ”αͶͻ (for EP14 and EP15)

TCOND9: –‘–ƒŽƒ”αͷͲ (for EP14 and EP15)

© ISO/IEC 2015 – All rights reserved 57

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TCOND10: –‘–ƒŽƒ”α͸ͻ (for EP13 and EP14)

TCOND11: –‘–ƒŽƒ”α͹Ͳ (for EP13 and EP14)

TCOND12: –‘–ƒŽƒ”αͳͲͲ (for EP13 and EP17)

For the invalid’ƒ”–‹–‹‘•‘ˆ–Š‡‹’—–ϐ‹‡Ž†•ǡ–Š‡ˆ‘ŽŽ‘™‹‰„‘—†ƒ”‹‡•…ƒ„‡‹†‡–‹ϐ‹‡†Ǥ

TCOND13: ‡šƒƒ”α͵ʹ͹͸͹ (for EP3)

TCOND14: ‡šƒƒ”αǦ͵ʹ͹͸ͺ (for EP4)

TCOND15: …‘—”•‡™‘”ƒ”α͵ʹ͹͸͹ (for EP5)

TCOND16: …‘—”•‡™‘”ƒ”αǦ͵ʹ͹͸ͺ (for EP6)

‹ƒŽŽ›ǡˆ‘”–Š‡invalid’ƒ”–‹–‹‘•‘ˆ–‘–ƒŽƒ”ǡ–Š‡ˆ‘ŽŽ‘™‹‰„‘—†ƒ”‹‡•…ƒ„‡‹†‡–‹ϐ‹‡†Ǥ

TCOND17: –‘–ƒŽƒ”αͳͲͳ (for EP17)

TCOND18: –‘–ƒŽƒ”α͵ʹ͹͸͹ (for EP17)

TCOND19: –‘–ƒŽƒ”αǦͳ (for EP18)

TCOND20: –‘–ƒŽƒ”αǦ͵ʹ͹͸ͺ (for EP18)

B.2.3.5 Step 3: Derive Test Coverage Items (TD3)

ˆ ͵Ǧ˜ƒŽ—‡ „‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹• ‹• ƒ’’Ž‹‡†ǡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ –Š‡ ˜ƒŽ—‡• –Šƒ– ƒ”‡ ‘ –Š‡
„‘—†ƒ”›‘ˆ–Š‡‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘ǡƒ†‡‹–Š‡”•‹†‡‘ˆ–Š‡„‘—†ƒ”›ǡ—•‹‰–Š‡•ƒŽŽ‡•–•‹‰‹ϐ‹…ƒ–
†‹•–ƒ…‡ƒ™ƒ›ǡƒ••Š‘™‹Figure B.9:
boundary boundary

Figure B.9 — Test coverage items for 3-value boundary value analysis

ͳ Ž–‡”ƒ–‹˜‡Ž›ǡ ʹǦ˜ƒŽ—‡ „‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹• …‘—Ž† ƒŽ•‘ „‡ ’‡”ˆ‘”‡†ǡ ™Š‹…Š ™‘—Ž† ”‡•—Ž– ‹ ƒ
•ƒŽŽ‡”—„‡”‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•„‡‹‰†‡”‹˜‡†ǡƒ†…‘””‡•’‘†‹‰Ž›ˆ‡™‡”–‡•–…ƒ•‡•„‡‹‰†‡”‹˜‡†Ǥ

‘” –Š‡ „‘—†ƒ”‹‡• –Šƒ– ™‡”‡ ‹†‡–‹ϐ‹‡† ‹ –Š‡ ’”‡˜‹‘—• •–‡’ ƒ• –‡•– …‘†‹–‹‘•ǡ –Š‡ ˆ‘ŽŽ‘™‹‰ –‡•–
…‘˜‡”ƒ‰‡‹–‡•ȋȌ…ƒ„‡‹†‡–‹ϐ‹‡†Ǥ•™‡ƒ”‡—•‹‰‹–‡‰‡”•‹–Š‹•‡šƒ’Ž‡ǡ–Š‡–‡•–…‘˜‡”ƒ‰‡
‹–‡•ƒ”‡‘‡‡‹–Š‡”•‹†‡‘ˆ‡ƒ…Š„‘—†ƒ”›Ǥ
ʹ ˆ –Š‡ ‡šƒ’Ž‡ ‹˜‘Ž˜‡† —‡”‹…ƒŽ †ƒ–ƒ –›’‡• ‹˜‘Ž˜‹‰ †‡…‹ƒŽ• ȋ‡Ǥ‰Ǥ ”‡ƒŽ —„‡”•Ȍǡ –Š‡ –Š‡ –‡•–
…‘˜‡”ƒ‰‡ ‹–‡• ˆ‘” „‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹• ™‘—Ž† „‡ –Š‡ •ƒŽŽ‡•– •‹‰‹ϐ‹…ƒ– ˜ƒŽ—‡ ˆ‘” –Š‡ †ƒ–ƒ –›’‡ —†‡”
consideration.

TCOVER1: ‡šƒƒ”αǦͳ (from TCOND1)

TCOVER2: ‡šƒƒ”αͲ (from TCOND1)

TCOVER3: ‡šƒƒ”αͳ (from TCOND1)

TCOVER4: ‡šƒƒ”α͹Ͷ (from TCOND2)

58 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOVER5: ‡šƒƒ”α͹ͷ (from TCOND2)

TCOVER6: ‡šƒƒ”α͹͸ (from TCOND2)

TCOVER7: …‘—”•‡™‘”ƒ”αǦͳ (from TCOND3)

TCOVER8: …‘—”•‡™‘”ƒ”αͲ (from TCOND3)

TCOVER9: …‘—”•‡™‘”ƒ”αͳ (from TCOND3)

TCOVER10: …‘—”•‡™‘”ƒ”αʹͶ (from TCOND4)

TCOVER11: …‘—”•‡™‘”ƒ”αʹͷ (from TCOND4)

TCOVER12: …‘—”•‡™‘”ƒ”αʹ͸ (from TCOND4)

‘”–Š‡‘—–’—–ϐ‹‡Ž†•ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡‹†‡–‹ϐ‹‡†ǣ

TCOVER13: –‘–ƒŽƒ”αǦͳ ȋˆ”‘ͷƬͳͻȌ

TCOVER14: –‘–ƒŽƒ”αͲ ȋˆ”‘ͷƬͳͻȌ

TCOVER15: –‘–ƒŽƒ”αͳ (from TCOND5)

TCOVER16: –‘–ƒŽƒ”αʹͺ (from TCOND6)

TCOVER17: –‘–ƒŽƒ”αʹͻ ȋˆ”‘͸Ƭ͹Ȍ

TCOVER18: –‘–ƒŽƒ”α͵Ͳ ȋˆ”‘͸Ƭ͹Ȍ

TCOVER19: –‘–ƒŽƒ”α͵ͳ (from TCOND7)

TCOVER20: –‘–ƒŽƒ”αͶͺ (from TCOND8)

TCOVER21: –‘–ƒŽƒ”αͶͻ ȋˆ”‘ͺƬͻȌ

TCOVER22: –‘–ƒŽƒ”αͷͲ ȋˆ”‘ͺƬͻȌ

TCOVER23: –‘–ƒŽƒ”αͷͳ (from TCOND9)

TCOVER24: –‘–ƒŽƒ”α͸ͺ (from TCOND10)

TCOVER25: –‘–ƒŽƒ”α͸ͻ ȋˆ”‘ͳͲƬͳͳȌ

TCOVER26: –‘–ƒŽƒ”α͹Ͳ ȋˆ”‘ͳͲƬͳͳȌ

TCOVER27: –‘–ƒŽƒ”α͹ͳ (from TCOND11)

TCOVER28: –‘–ƒŽƒ”αͻͻ (from TCOND12)

TCOVER29: –‘–ƒŽƒ”αͳͲͲ ȋˆ”‘ͳʹƬͳ͹Ȍ

TCOVER30: –‘–ƒŽƒ”αͳͲͳ ȋˆ”‘ͳʹƬͳ͹Ȍ

‘–‡–Šƒ–‡“—‹˜ƒŽ‡…‡…Žƒ••‡•͹–‘ͳʹŠƒ˜‡‘–„‡‡…‘˜‡”‡†„›ƒ›–‡•–…‘˜‡”ƒ‰‡‹–‡•ǡ•‹…‡–Š‡›
Šƒ˜‡‘‹†‡–‹ϐ‹ƒ„Ž‡„‘—†ƒ”‹‡•ȋƒ••–ƒ–‡†‹–Š‡’”‡˜‹‘—••–‡’ȌǤ
‘” –Š‡ ”‡ƒ‹‹‰ ‹˜ƒŽ‹† ’ƒ”–‹–‹‘• –Šƒ– ™‡”‡ ‹†‡–‹ϐ‹‡† ȋ‹Ǥ‡Ǥ ˆ‘” –Š‡ „‘—†ƒ”‹‡• ‘ˆ –‡•– …‘†‹–‹‘•
ͳ͵–‘ʹͲ–Šƒ–Šƒ˜‡‘–›‡–„‡‡…‘˜‡”‡†Ȍǡ–Š‡ˆ‘ŽŽ‘™‹‰‹˜ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ
„‡‹†‡–‹ϐ‹‡†ǣ

TCOVER31: ‡šƒƒ”α͵ʹ͹͸͸ (from TCOND13)

© ISO/IEC 2015 – All rights reserved 59

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TCOVER32: ‡šƒƒ”α͵ʹ͹͸͹ (from TCOND13)

TCOVER33: ‡šƒƒ”α͵ʹ͹͸ͺ (from TCOND13)

TCOVER34: ‡šƒƒ”αǦ͵ʹ͹͸ͻ (from TCOND14)

TCOVER35: ‡šƒƒ”αǦ͵ʹ͹͸ͺ (from TCOND14)

TCOVER36: ‡šƒƒ”αǦ͵ʹ͹͸͹ (from TCOND14)

TCOVER37: …‘—”•‡™‘”ƒ”α͵ʹ͹͸͸ (from TCOND15)

TCOVER38: …‘—”•‡™‘”ƒ”α͵ʹ͹͸͹ (from TCOND15)

TCOVER39: …‘—”•‡™‘”ƒ”α͵ʹ͹͸ͺ (from TCOND15)

TCOVER40: …‘—”•‡™‘”ƒ”αǦ͵ʹ͹͸ͻ (from TCOND16)

TCOVER41: …‘—”•‡™‘”ƒ”αǦ͵ʹ͹͸ͺ (from TCOND16)

TCOVER42: …‘—”•‡™‘”ƒ”αǦ͵ʹ͹͸͹ (from TCOND16)

TCOVER43: –‘–ƒŽƒ”αͳͲʹ (from TCOND17)

TCOVER44: –‘–ƒŽƒ”α͵ʹ͹͸͸ (from TCOND18)

TCOVER45: –‘–ƒŽƒ”α͵ʹ͹͸͹ (from TCOND18)

TCOVER46: –‘–ƒŽƒ”α͵ʹ͹͸ͺ (from TCOND18)

TCOVER47: –‘–ƒŽƒ”αǦʹ (from TCOND19)

TCOVER48: –‘–ƒŽƒ”αǦ͵ʹ͹͸ͻ (from TCOND20)

TCOVER49: –‘–ƒŽƒ”αǦ͵ʹ͹͸ͺ (from TCOND20)

TCOVER50: –‘–ƒŽƒ”αǦ͵ʹ͹͸͹ (from TCOND20)

B.2.3.6 Step 4: Derive Test Cases (TD4)

Test cases can now be derived to cover the required percentage of test coverage items that were
‹†‡–‹ϐ‹‡† ‹ –Š‡ ’”‡˜‹‘—• •–‡’Ǥ ‘” ‡šƒ’Ž‡ǡ ‹ˆ ͳͲͲΨ „‘—†ƒ”› …‘˜‡”ƒ‰‡ ‹• ”‡“—‹”‡†ǡ –Š‡ –‡•– …ƒ•‡•
—•– „‡ †‡”‹˜‡† –‘ …‘˜‡” ƒŽŽ –‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ǥ ‡Ǧ–‘Ǧ‘‡ „‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹• …‘—Ž† „‡ —•‡†
–‘†‡”‹˜‡‘‡–‡•–…ƒ•‡’‡”–‡•–…‘˜‡”ƒ‰‡‹–‡ǡ‘”‹‹‹œ‡†„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•…‘—Ž†„‡—•‡†–‘
derive the minimum number of test cases required to cover all test coverage items.
The preconditions of all test cases for the generate_grading function are the same: the application is
”‡ƒ†›–‘–ƒ‡–Š‡‹’—–•‘ˆ‡šƒƒ†…‘—”•‡™‘”ƒ”Ǥ
ˆ™‡ƒ••—‡–Šƒ–ͳͲͲΨ„‘—†ƒ”›…‘˜‡”ƒ‰‡‹•”‡“—‹”‡†ƒ†–Šƒ–‘‡Ǧ–‘Ǧ‘‡„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•
is used to derive test cases, then six test cases can be derived for the input exam mark, as shown in
Table B.12Ǥƒ…Š–‡•–…ƒ•‡‹•†‡”‹˜‡†ƒ•ˆ‘ŽŽ‘™•ǣϐ‹”•–ǡ•‡Ž‡…–‹‰‘‡„‘—†ƒ”›˜ƒŽ—‡ȋ–‡•–…‘˜‡”ƒ‰‡‹–‡Ȍ
ˆ‘”‹…Ž—•‹‘‹‡ƒ…Š–‡•–…ƒ•‡Ǣ•‡…‘†Ž›ǡƒŽŽ‘…ƒ–‹‰ƒƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡–‘ƒŽŽ‘–Š‡”‹’—–•’”‡•‡–‹
the test case; and third, determining the expected result of the test.

Table B.12 — Boundary values for exam mark

Test Case 1 2 3 4 5 6
Input (exam mark) -1 0 1 74 75 76
Input (c/w mark) 15 15 15 15 15 15

60 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.12 (continued)

Test Case 1 2 3 4 5 6
total mark (as calculated) 14 15 16 89 90 91
Test Coverage Item 1 2 3 4 5 6
‘—†ƒ”›–‡•–‡†ȋ‡šƒƒ”Ȍ 0 75
Exp. Output Ǯ ǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯ ǯ

‘–‡–Šƒ––Š‡‹’—–…‘—”•‡™‘”ȋ…Ȁ™Ȍƒ”Šƒ•„‡‡•‡––‘ƒƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡‘ˆͳͷǡƒ•–Š‡•‡–‡•–
cases are focused on exercising the input exam mark boundaries.
And the test cases derived from the input coursework mark are:

Table B.13 — Boundary values for coursework mark

Test Case 7 8 9 10 11 12
Input (exam mark) 40 40 40 40 40 40
Input (c/w mark) -1 0 1 24 25 26
total mark (as calculated) 39 40 41 64 65 66
Test Coverage Item 7 8 9 10 11 12
‘—†ƒ”›–‡•–‡†ȋ…Ȁ™ƒ”Ȍ 0 25
Exp. Output Ǯ ǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯ ǯ

‘–‡–Šƒ––Š‡‹’—–‡šƒƒ”Šƒ•„‡‡•‡––‘ƒƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡‘ˆͶͲǤ
The test cases derived from the outputs are:

Table B.14 — Test cases for total mark

Test Case 13 14 15 16 17 18 19
Input (exam mark) -1 0 0 28 29 15 6
Input (c/w mark) 0 0 1 0 0 15 25
total mark (as calculated) -1 0 1 28 29 30 31
Test Coverage Item 13 14 15 16 17 18 19
‘—†ƒ”›–‡•–‡†ȋ–‘–ƒŽƒ”Ȍ 0 29 29, 30 30
Exp. Output Ǯ ǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ

Table B.15 — Test cases for total mark

Test Case 20 21 22 23 24 25 26 27
Input (exam mark) 23 24 50 26 48 49 45 71
Input (c/w mark) 25 25 0 25 20 20 25 0
‘–ƒŽƒ”ȋƒ•…ƒŽ…—Žƒ–‡†Ȍ 48 49 50 51 68 69 70 71
Test Coverage Item 20 21 22 23 24 25 26 27
‘—†ƒ”›–‡•–‡†ȋ–‘–ƒŽƒ”Ȍ 49 49, 50 50 69 69, 70 70
Exp. Output Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ Ǯǯ

© ISO/IEC 2015 – All rights reserved 61

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.16 — Test cases for total mark

Test Case 28 29 30
Input (exam mark) 74 75 75
Input (c/w mark) 25 25 26
‘–ƒŽƒ”ȋƒ•…ƒŽ…—Žƒ–‡†Ȍ 99 100 101
Test Coverage Item 28 29 30
‘—†ƒ”›–‡•–‡†ȋ–‘–ƒŽƒ”Ȍ 100 100, 101
Exp. Output Ǯǯ Ǯǯ Ǯ ǯ

The input values of exam mark and coursework mark have been derived from total mark, which is their
sum.
The following test cases are required to cover the remaining test coverage items TCOVER31 to TCOVER50,
™Š‹…Š™‡”‡‹†‡–‹ϐ‹‡†‘—–•‹†‡–Š‡˜ƒŽ‹†„‘—†ƒ”‹‡•ƒ†ƒ––Š‡‡š–”‡‡‡†‰‡•‘ˆ‡“—‹˜ƒŽ‡…‡…Žƒ••‡•ǣ

Table B.17 — Test cases for exam mark

Test Case 31 32 33 34 35 36
Input (exam mark) 32766 32767 32768 -32769 -32768 -32767
Input (c/w mark) 15 15 15 15 15 15
total mark (as calculated) 32781 32782 32783 -32754 -32753 -32752
Test Coverage Item 31 32 33 34 35 36
‘—†ƒ”›–‡•–‡†ȋ‡šƒƒ”Ȍ 32767 -32768
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table B.18 — Test cases for coursework mark

Test Case 37 38 39 40 41 42
Input (exam mark) 40 40 40 40 40 40
Input (c/w mark) 32766 32767 32768 -32769 -32768 -32767
total mark (as calculated) 32806 32807 32808 -32729 -32728 -32727
Test Coverage Item 37 38 39 40 41 42
‘—†ƒ”›–‡•–‡†ȋ…Ȁ™ƒ”Ȍ 32767 -32768
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table B.19 — Test cases for total mark

Test Case 43 44 45 46 47 48 49 50
Input (exam mark) 75 16383 32767 1 -1 0 -16384 -32766
Input (c/w mark) 27 16383 0 32767 -1 -32769 -16384 -1
‘–ƒŽƒ”ȋƒ•…ƒŽ…—Žƒ–‡†Ȍ 102 32766 32767 32768 -2 -32769 -32768 -32767
Test Coverage Item 43 44 45 46 47 48 49 50
‘—†ƒ”›–‡•–‡†ȋ–‘–ƒŽƒ”Ȍ 101 32767 1 -32768
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

It should be noted that when invalid input values are used (as above, in test cases 1, 6, 7, 12, 13, and
͵ͲǦͷͲȌ ‹– ƒ›ǡ †‡’‡†‹‰ ‘ –Š‡ ‹’Ž‡‡–ƒ–‹‘ǡ „‡ ‹’‘••‹„Ž‡ –‘ ƒ…–—ƒŽŽ› ‡š‡…—–‡ –Š‡ –‡•– …ƒ•‡Ǥ ‘”
instance, in the Ada programming language, if the input variable is declared as a positive integer then it

62 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

will not be possible to assign a negative value to it. Despite this, it is still worthwhile considering all the
test cases for completeness.
Š‡ƒ„‘˜‡–‡•–…ƒ•‡•—‹–‡ƒ…Š‹‡˜‡•ͳͲͲΨ„‘—†ƒ”›˜ƒŽ—‡…‘˜‡”ƒ‰‡ˆ‘”͵Ǧ˜ƒŽ—‡„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•
ƒ• ‹– ‡ƒ„Ž‡• ƒŽŽ ‹†‡–‹ϐ‹‡† –‡•– …‘˜‡”ƒ‰‡ ‹–‡• –‘ „‡ ‡š‡”…‹•‡† „› ƒ– Ž‡ƒ•– ‘‡ –‡•– …ƒ•‡Ǥ ‘™‡” Ž‡˜‡Ž•
‘ˆ…‘˜‡”ƒ‰‡™‘—Ž†„‡ƒ…Š‹‡˜‡†‹ˆ•‘‡‘ˆ–Š‡‹†‡–‹ϐ‹‡†„‘—†ƒ”‹‡•™‡”‡‘–‡š‡”…‹•‡†Ǥˆ•‘‡‘ˆ–Š‡
„‘—†ƒ”‹‡•ƒ”‡‘–‹†‡–‹ϐ‹‡†ǡ–Š‡ƒ›…‘˜‡”ƒ‰‡‡ƒ•—”‡„ƒ•‡†‘–Š‹•‹…‘’Ž‡–‡•‡–‘ˆ„‘—†ƒ”‹‡•
would be misleading.

B.2.3.7 Step 5: Assemble Test Sets (TD5)

ˆ™‡ƒ••—‡–Šƒ–‹–‹•’‘••‹„Ž‡–‘ƒ—–‘ƒ–‹…ƒŽŽ›…Š‡…ƒƒ……‡’–Ȁ”‡Œ‡…–”‡•’‘•‡ˆ‘”‡ƒ…Š˜ƒŽ‹†–‡•–…ƒ•‡ǡ
„—––Šƒ–ƒ—–‘ƒ–‹‘…ƒ‘–Šƒ†Ž‡ˆƒ—Ž–‡••ƒ‰‡•ȋ Ȍ–Šƒ–ƒ”‡‰‡‡”ƒ–‡†„›‹˜ƒŽ‹†–‡•–…ƒ•‡•ǡ–Š‡
we can generate two test sets (TS); one for manual testing and one for automated testing.

ͳǣƒ—ƒŽ‡•–‹‰Ȃͳǡ͸ǡ͹ǡͳʹǡͳ͵ǡ͵Ͳ–‘ͷͲǤ

TS2: Automated Testing – TEST CASES 2, 3, 4, 5, 8, 9, 10, 11 and 14 to 29.

B.2.3.8 Step 6: Derive Test Procedures (TD6)

‘”–Š‡ƒ—ƒŽ–‡•–…ƒ•‡•‹–‡•–•‡–ͳǡ‘‡–‡•–’”‘…‡†—”‡ȋȌ…ƒ„‡†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•ǣ
ͳǣƒ—ƒŽ–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ
For the automated test cases in test set TS2, one test script could be written to execute all test cases in
the test set, as follows:
ʹǣƒ—–‘ƒ–‡†–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ
For the automated test procedure TP2, automation code to execute the procedure would need to be
written.

B.2.3.9 Boundary Value Analysis Coverage

Using the formula provided in 6.2.3 and the test coverage items derived above:
50
Coverage( boundary _ value _ analysis ) = × 100% = 100%
50
Š—•ǡͳͲͲΨ…‘˜‡”ƒ‰‡‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•Šƒ•„‡‡ƒ…Š‹‡˜‡†Ǥ

B.2.4 Syntax Testing

B.2.4.1 Introduction

Š‡ ƒ‹ ‘ˆ •›–ƒš –‡•–‹‰ ‹• –‘ †‡”‹˜‡ ƒ •‡– ‘ˆ –‡•– …ƒ•‡• –Šƒ– …‘˜‡” –Š‡ ‹’—– •›–ƒš ‘ˆ –Š‡ –‡•– ‹–‡
ƒ……‘”†‹‰–‘–Š‡…Š‘•‡Ž‡˜‡Ž‘ˆ‹’—–•›–ƒš…‘˜‡”ƒ‰‡ǤŠ‹•–‡…Š‹“—‡‹•„ƒ•‡†—’‘ƒƒƒŽ›•‹•‘ˆ–Š‡
–‡•–„ƒ•‹•‘ˆ–Š‡–‡•–‹–‡–‘‘†‡Ž‹–•„‡Šƒ˜‹‘—”„›‡ƒ•‘ˆƒ†‡•…”‹’–‹‘‘ˆ–Š‡‹’—–˜‹ƒ‹–••›–ƒšǤ
‡‹ŽŽ—•–”ƒ–‡–Š‡–‡…Š‹“—‡„›‡ƒ•‘ˆƒ™‘”‡†‡šƒ’Ž‡ǤŠ‡–‡…Š‹“—‡‹•‘Ž›‡ˆˆ‡…–‹˜‡–‘–Š‡‡š–‡–
–Šƒ––Š‡•›–ƒšƒ•†‡ϐ‹‡†…‘””‡•’‘†•–‘–Š‡”‡“—‹”‡†•›–ƒšǤ

ǤʹǤͶǤʹ ’‡…‹ϐ‹…ƒ–‹‘

‘•‹†‡”ƒ–‡•–‹–‡–Šƒ–•‹’Ž›…Š‡…•™Š‡–Š‡”ƒ‹’—–ϔŽ‘ƒ–̸‹…‘ˆ‘”•–‘–Š‡•›–ƒš‘ˆƒϐŽ‘ƒ–‹‰
’‘‹–—„‡”ǡϐŽ‘ƒ–ȋ†‡ϐ‹‡†„‡Ž‘™ȌǤŠ‡–‡•–‹–‡‘—–’—–•check_res, which takes the form “valid” or
“invalid” dependent on the result of its check.
‡”‡‹•ƒ”‡’”‡•‡–ƒ–‹‘‘ˆ–Š‡•›–ƒšˆ‘”–Š‡ϐŽ‘ƒ–‹‰’‘‹–—„‡”ǡÀRDW in Backus Naur Form (BNF):

© ISO/IEC 2015 – All rights reserved 63

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

ÀRDW LQW³H´LQW
LQW >³´_´´@QDW
QDW ^GLJ`
GLJ ³´_´´_´´_´´_´´_´´_´´_´´_´´_´´

‡”‹ƒŽ•ƒ”‡•Š‘™‹“—‘–ƒ–‹‘ƒ”•Ǣ–Š‡•‡ƒ”‡–Š‡‘•–‡Ž‡‡–ƒ”›’ƒ”–•‘ˆ–Š‡•›–ƒšȂ–Š‡ƒ…–—ƒŽ
characters that make up the input to the test item. _ separates alternatives. >@ surrounds an optional
item, that is, one for which nothing is an alternative. ^`•—””‘—†•ƒ‹–‡™Š‹…Šƒ›„‡‹–‡”ƒ–‡†‘‡
or more times.

B.2.4.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: ÀRDWBLQ

B.2.4.4 Step 2: Derive Test Conditions (TD2)

Š‡ϐ‹”•–•–‡’‹•–‘†‡”‹˜‡–Š‡–‡•–…‘†‹–‹‘•ˆ”‘–Š‡•›–ƒšǤ‡•–…‘†‹–‹‘•ƒ›„‡†‡ϐ‹‡†ƒ•–Š‡
‹’—–’ƒ”ƒ‡–‡”•‹–Š‡•›–ƒšǡƒ•ˆ‘ŽŽ‘™•ǣ

TCOND1 ϐŽ‘ƒ–ዐ–Dz‡dz‹–

TCOND2 ‹–αȏDzΪdzȁdzǦdzȐƒ–

TCOND3 ƒ–αȓ†‹‰Ȕ

TCOND4 †‹‰αDzͲdzȁdzͳdzȁdzʹdzȁdz͵dzȁdzͶdzȁdzͷdzȁdz͸dzȁdz͹dzȁdzͺdzȁdzͻdz

B.2.4.5 Step 3: Derive Test Coverage Items (TD3)

Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”•›–ƒš–‡•–‹‰ƒ”‡–Š‡Dz‘’–‹‘•dzȋ˜ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡‹–‡•Ȍƒ†Dz—–ƒ–‹‘•dz
ȋ‹˜ƒŽ‹† –‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ȍ ‘ˆ –Š‡ †‡ϐ‹‡† •›–ƒš ȋ•‡‡ 5.2.4.2 ˆ‘” †‡ϐ‹‹–‹‘• ‘ˆ Dz‘’–‹‘•dz ƒ†
“mutations”).
ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡†‡”‹˜‡†ˆ‘”–Š‡‡Ž‡‡–•‘–Š‡”‹‰Š–Šƒ†•‹†‡‘ˆ–Š‡ †‡ϐ‹‹–‹‘Ǥ
There are three test coverage items that can be derived for the “+” and “-” signs of TCOND2:

TCOVER1: there is no “+” or “-” sign ȋˆ‘”ʹƬͳȌ

TCOVER2: there is a “+” sign ȋˆ‘”ʹƬͳȌ

TCOVER3: there is a “-” sign ȋˆ‘”ʹƬͳȌ

ͳ ‡’ƒ”ƒ–‡–‡•–…‘˜‡”ƒ‰‡‹–‡•…‘—Ž†„‡†‡”‹˜‡†ˆ‘”–Š‡ϐ‹”•–ƒ†•‡…‘†‹•–ƒ…‡•‘ˆDzΪdzƒ†DzǦdz‹ˆ”‡“—‹”‡†Ǥ

QDW has two test coverage items:

TCOVER4: nat is a single digit number ȋˆ‘”͵ƬʹȌ

TCOVER5: nat is a multiple digit number ȋˆ‘”͵ƬʹȌ

ʹ ‡’ƒ”ƒ–‡–‡•–…‘˜‡”ƒ‰‡‹–‡•…‘—Ž†„‡†‡”‹˜‡†ˆ‘”–Š‡ϐ‹”•–ƒ†•‡…‘†‹•–ƒ…‡•‘ˆƒ–‹ˆ”‡“—‹”‡†Ǥ

GLJ has ten options:

TCOVER6: integer is a “0” ȋˆ‘”ͶƬ͵Ȍ

TCOVER7: integer is a “1” ȋˆ‘”ͶƬ͵Ȍ

64 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOVER8: integer is a “2” ȋˆ‘”ͶƬ͵Ȍ

TCOVER9: integer is a “3” ȋˆ‘”ͶƬ͵Ȍ

TCOVER10: integer is a “4” ȋˆ‘”ͶƬ͵Ȍ

TCOVER11: integer is a “5” ȋˆ‘”ͶƬ͵Ȍ

TCOVER12: integer is a “6” ȋˆ‘”ͶƬ͵Ȍ

TCOVER13: integer is a “7” ȋˆ‘”ͶƬ͵Ȍ

TCOVER14: integer is an “8” ȋˆ‘”ͶƬ͵Ȍ

TCOVER15: integer is a “9” ȋˆ‘”ͶƬ͵Ȍ

Š‡”‡ƒ”‡–Š—•ϐ‹ˆ–‡‡˜ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–…ƒ„‡†‡ϐ‹‡†Ǥ
Š‡ϐ‹”•–•–‡’‹†‡”‹˜‹‰‹˜ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡‹–‡•‹•–‘…‘•–”—…–ƒ…Š‡…Ž‹•–‘ˆ‰‡‡”‹…—–ƒ–‹‘•–Šƒ–
can be applied to the test conditions. A possible checklist is:

m1. introduce an invalid value for an element;

ʹǤ•—„•–‹–—–‡ƒ‡Ž‡‡–™‹–Šƒ‘–Š‡”†‡ϐ‹‡†‡Ž‡‡–Ǣ

͵Ǥ‹••‘—–ƒ†‡ϐ‹‡†‡Ž‡‡–Ǣ

m4. add an extra element.

͵ –Š‡” –›’‡• ‘ˆ •›–ƒš —–ƒ–‹‘ …‘—Ž† „‡ —•‡†ǡ †‡’‡†‹‰ ‘ –Š‡ –›’‡• ‘ˆ †‡ˆ‡…–• –‡•–‹‰ ‹•
intending to target.

Š‡•‡‰‡‡”‹…—–ƒ–‹‘•ƒ”‡ƒ’’Ž‹‡†–‘–Š‡‹†‹˜‹†—ƒŽ‡Ž‡‡–•‘ˆ–Š‡•›–ƒš–‘›‹‡Ž†•’‡…‹ϐ‹…—–ƒ–‹‘•Ǥ

TCOVER16: ƒ’’Ž›ͳ–‘ϐ‹”•–DzLQW” (for TCOND1)

TCOVER17: ƒ’’Ž›ͳ–‘DzH” (for TCOND1)

TCOVER18: ƒ’’Ž›ͳ–‘•‡…‘†DzLQW” (for TCOND1)

TCOVER19: ƒ’’Ž›ͳ–‘Dz>“”_””@” (for TCOND2)

TCOVER20: ƒ’’Ž›ͳ–‘DzQDW” (for TCOND2)

TCOVER21: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡DzHdzˆ‘”ϐ‹”•–DzLQW” (for TCOND1)

TCOVER22: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡Dz>“”_””@dzϐ‹”•–DzLQW” ȋˆ‘”ͳƬʹȌ

TCOVER23: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡ϐ‹”•–DzLQW” for “H” (for TCOND1)

TCOVER24: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡Dz>“”_””@” for “H” ȋˆ‘”ͳƬʹȌ

TCOVER25: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡DzH” for second “LQW” (for TCOND1)

TCOVER26: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡Dz>“”_””@” for second “LQW” (for TCOND1)

TCOVER27: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡DzH” for “>“”_””@” ȋˆ‘”ͳƬʹȌ

TCOVER28: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡DzH” for “QDW” ȋˆ‘”ͳƬʹȌ

TCOVER29: ƒ’’Ž›ʹ–‘•—„•–‹–—–‡Dz>“”_””@” for “QDW” ȋˆ‘”ͳƬʹȌ

© ISO/IEC 2015 – All rights reserved 65

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TCOVER30: ƒ’’Ž›͵–‘ϐ‹”•–DzLQW” (for TCOND1)

TCOVER31: ƒ’’Ž›͵–‘DzH” (for TCOND1)

TCOVER32: ƒ’’Ž›͵–‘•‡…‘†DzLQW” (for TCOND1)

TCOVER33: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡ˆ‘”‡ϐ‹”•–DzLQW” (for TCOND1)

TCOVER34: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡ˆ‘”‡DzH” (for TCOND1)

TCOVER35: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡ˆ‘”‡•‡…‘†DzLQW” (for TCOND1)

TCOVER36: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–ƒˆ–‡”•‡…‘†DzLQW” (for TCOND1)

TCOVER37: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡ˆ‘”‡ϐ‹”•–DzLQW” and “>“”_””@” ȋˆ‘”ͳƬʹȌ

TCOVER38: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡–™‡‡DzȏDzΪdzȁdzǦdzȐdzƒ†ϐ‹”•–Dz‹–dz ȋˆ‘”ͳƬʹȌ

TCOVER39: ƒ’’Ž›Ͷ–‘ƒ††‡Ž‡‡–„‡–™‡‡ϐ‹”•–Dz‹–dzƒ†Dz‡dz (for TCOND1)

ȏDzΪdzȁdzǦdzȐ Šƒ• „‡‡ –”‡ƒ–‡† ƒ• ƒ •‹‰Ž‡ ‡Ž‡‡– „‡…ƒ—•‡ –Š‡ —–ƒ–‹‘ ‘ˆ ‹†‹˜‹†—ƒŽ ‘’–‹‘ƒŽ ‹–‡•
•‡’ƒ”ƒ–‡Ž›†‘‡•‘–…”‡ƒ–‡–‡•–…ƒ•‡•™‹–Š‹˜ƒŽ‹†•›–ƒšȋ—•‹‰–Š‡•‡‰‡‡”‹…—–ƒ–‹‘•ȌǤ

B.2.4.6 Step 4: Derive Test Cases (TD4)

ƒŽ‹† –‡•– …ƒ•‡• ƒ”‡ †‡”‹˜‡† „› •‡Ž‡…–‹‰ ‘‡ ‘” ‘”‡ ‘’–‹‘• ˆ‘” ‹…Ž—•‹‘ ‹ –Š‡ …—””‡– –‡•– …ƒ•‡ǡ
‹†‡–‹ˆ›‹‰‹’—–•–‘‡š‡”…‹•‡–Š‡‘’–‹‘ȋ•Ȍƒ††‡–‡”‹‹‰–Š‡‡š’‡…–‡†”‡•—Ž–ȋ‹–Š‹•…ƒ•‡ǡǮ…Š‡…̴
res’). The resulting valid test cases are:

Table B.20 — Valid test cases for syntax testing

Test Case ’—–ǮϐŽ‘ƒ–̴‹ǯ Test Coverage Item š’‡…–‡†‡•—Ž–Ǯ…Š‡…̴”‡•ǯ

TC 1 3e2 TCOVER1 Ǯ˜ƒŽ‹†ǯ
TC 2 +2e+5 TCOVER2 Ǯ˜ƒŽ‹†ǯ
TC 3 -6e-7 TCOVER3 Ǯ˜ƒŽ‹†ǯ
TC 4 6e-2 TCOVER4 Ǯ˜ƒŽ‹†ǯ
TC 5 1234567890e3 TCOVER5 Ǯ˜ƒŽ‹†ǯ
TC 6 0e0 TCOVER6 Ǯ˜ƒŽ‹†ǯ
TC 7 1e1 TCOVER7 Ǯ˜ƒŽ‹†ǯ
TC 8 2e2 TCOVER8 Ǯ˜ƒŽ‹†ǯ
TC 9 3e3 TCOVER9 Ǯ˜ƒŽ‹†ǯ
TC 10 4e4 TCOVER10 Ǯ˜ƒŽ‹†ǯ
TC 11 5e5 TCOVER11 Ǯ˜ƒŽ‹†ǯ
TC 12 6e6 TCOVER12 Ǯ˜ƒŽ‹†ǯ
TC 13 7e7 TCOVER13 Ǯ˜ƒŽ‹†ǯ
TC 14 8e8 TCOVER14 Ǯ˜ƒŽ‹†ǯ
TC 15 9e9 TCOVER15 Ǯ˜ƒŽ‹†ǯ

Š‹•‹•„›‘‡ƒ•ƒ‹‹ƒŽ–‡•–•‡––‘‡š‡”…‹•‡–Š‡ͳͷ‘’–‹‘•ȋ‹–…ƒ„‡”‡†—…‡†–‘Œ—•––Š”‡‡–‡•–
cases, for example, 2, 3 and 5 above), and some test cases will exercise more options than the single
‘‡ Ž‹•–‡† ‹ –Š‡ Dz‡•– ‘˜‡”ƒ‰‡ –‡dz …‘Ž—Ǥ ƒ…Š ‘’–‹‘ Šƒ• „‡‡ –”‡ƒ–‡† •‡’ƒ”ƒ–‡Ž› Š‡”‡ –‘ ƒ‹†
—†‡”•–ƒ†‹‰‘ˆ–Š‡‹”†‡”‹˜ƒ–‹‘ǤŠ‹•ƒ’’”‘ƒ…Šƒ›ƒŽ•‘…‘–”‹„—–‡–‘–Š‡‡ƒ•‡™‹–Š™Š‹…Š–Š‡…ƒ—•‡•
of failures are located.

66 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

˜ƒŽ‹†–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰‘‡‘”‘”‡—–ƒ–‹‘•ˆ‘”‹…Ž—•‹‘‹–Š‡…—””‡––‡•–…ƒ•‡ǡ
‹†‡–‹ˆ›‹‰‹’—–•–‘‡š‡”…‹•‡–Š‡—–ƒ–‹‘ȋ•Ȍƒ††‡–‡”‹‹‰–Š‡‡š’‡…–‡†”‡•—Ž–ȋ‹–Š‹•…ƒ•‡ǡǮ…Š‡…̴
res’). The resulting invalid test cases are:

Table B.21 — Invalid test cases for syntax testing

Test Case ’—–ǮϐŽ‘ƒ–̴‹ǯ —–ƒ–‹‘ Test Coverage Item š’‡…–‡†‡•—Ž–Ǯ…Š‡…̴”‡•ǯ

TC 16 xe0 m1 TCOVER16 Ǯ‹˜ƒŽ‹†ǯ
TC 17 0x0 m1 TCOVER17 Ǯ‹˜ƒŽ‹†ǯ
TC 18 0ex m1 TCOVER18 Ǯ‹˜ƒŽ‹†ǯ
TC 19 x0e0 m1 TCOVER19 Ǯ‹˜ƒŽ‹†ǯ
TC 20 +xe0 m1 TCOVER20 Ǯ‹˜ƒŽ‹†ǯ
TC 21 ee0 m2 TCOVER21 Ǯ‹˜ƒŽ‹†ǯ
TC 22 +e0 m2 TCOVER22 Ǯ‹˜ƒŽ‹†ǯ
TC 23 000 m2 TCOVER23 Ǯ‹˜ƒŽ‹†ǯ
TC 24 0+0 m2 TCOVER24 Ǯ‹˜ƒŽ‹†ǯ
TC 25 0ee m2 TCOVER25 Ǯ‹˜ƒŽ‹†ǯ
TC 26 0e+ m2 TCOVER26 Ǯ‹˜ƒŽ‹†ǯ
TC 27 e0e0 m2 TCOVER27 Ǯ‹˜ƒŽ‹†ǯ
TC 28 +ee0 m2 TCOVER28 Ǯ‹˜ƒŽ‹†ǯ
TC 29 ++e0 m2 TCOVER29 Ǯ‹˜ƒŽ‹†ǯ
TC 30 e0 m3 TCOVER30 Ǯ‹˜ƒŽ‹†ǯ
TC 31 00 m3 TCOVER31 Ǯ‹˜ƒŽ‹†ǯ
TC 32 0e m3 TCOVER32 Ǯ‹˜ƒŽ‹†ǯ
TC 33 ›Ͳ‡Ͳ m4 TCOVER33 Ǯ‹˜ƒŽ‹†ǯ
TC 34 Ͳ›‡Ͳ m4 TCOVER34 Ǯ‹˜ƒŽ‹†ǯ
TC 35 Ͳ‡›Ͳ m4 TCOVER35 Ǯ‹˜ƒŽ‹†ǯ
TC 36 Ͳ‡Ͳ› m4 TCOVER36 Ǯ‹˜ƒŽ‹†ǯ
TC 37 ›ΪͲ‡Ͳ m4 TCOVER37 Ǯ‹˜ƒŽ‹†ǯ
TC 38 Ϊ›Ͳ‡Ͳ m4 TCOVER38 Ǯ‹˜ƒŽ‹†ǯ
TC 39 ΪͲ›‡Ͳ m4 TCOVER39 Ǯ‹˜ƒŽ‹†ǯ

‘‡ ‘ˆ –Š‡ —–ƒ–‹‘• ƒ”‡ ‹†‹•–‹‰—‹•Šƒ„Ž‡ ˆ”‘ …‘””‡…–Ž› ˆ‘”‡† ‡š’ƒ•‹‘• ƒ† –Š‡•‡ Šƒ˜‡ „‡‡
discarded. For example, the generic mutation m2 (substitute TCOND2 for TCOND4) generates correct
•›–ƒšƒ•ʹ‹•Dz•—„•–‹–—–‡ƒ‡Ž‡‡–™‹–Šƒ‘–Š‡”†‡ϐ‹‡†‡Ž‡‡–dzƒ†ʹƒ†Ͷƒ”‡
the same (int).
‘‡ ‘ˆ –Š‡ ”‡ƒ‹‹‰ —–ƒ–‹‘• ƒ”‡ ‹†‹•–‹‰—‹•Šƒ„Ž‡ ˆ”‘ ‡ƒ…Š ‘–Š‡” ƒ† –Š‡•‡ ƒ”‡ …‘˜‡”‡† „› ƒ
•‹‰Ž‡ –‡•– …ƒ•‡Ǥ ‘” ‡šƒ’Ž‡ǡ ƒ’’Ž›‹‰ –Š‡ ‰‡‡”‹… —–ƒ–‹‘ ͳ ȋDz‹–”‘†—…‡ ƒ ‹˜ƒŽ‹† ˜ƒŽ—‡ ˆ‘” ƒ
‡Ž‡‡–dzȌ„›”‡’Žƒ…‹‰Ͷǡ™Š‹…Š•Š‘—Ž†„‡ƒ‹–‡‰‡”ǡ™‹–ŠDzΪdz…”‡ƒ–‡•–Š‡ˆ‘”DzͲ‡ΪdzǤŠ‹•‹•–Š‡
same input as generated for test case 26 from Table B.21.
ƒ› ‘”‡ –‡•– …ƒ•‡• …ƒ „‡ …”‡ƒ–‡† „› ƒ‹‰ †‹ˆˆ‡”‡– …Š‘‹…‡• ™Š‡ —•‹‰ •‹‰Ž‡ —–ƒ–‹‘•ǡ ‘”
combining mutations.

B.2.4.7 Step 5: Assemble Test Sets (TD5)

†‡…‹•‹‘ƒ›„‡ƒ†‡–‘ƒ••‡„Ž‡‘‡–‡•–•‡–ˆ‘”˜ƒŽ‹†–‡•–…ƒ•‡•ƒ†‘‡ˆ‘”‹˜ƒŽ‹†–‡•–…ƒ•‡•ǣ

TS1: TEST CASE 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15

© ISO/IEC 2015 – All rights reserved 67

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TS2: TEST CASE 16, 17, 18, 19, 20, 21, 22, 23, 24, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39

B.2.4.8 Step 6: Derive Test Procedures (TD6)

All test cases could be assembled into the one test procedure, starting with the valid test cases, and
ending with the invalid test cases.

ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡˆ‘ŽŽ‘™‡†„›ƒŽŽ–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡
test sets.

B.2.4.9 Syntax Testing Coverage

As stated in 6.2.4ǡ–Š‡”‡‹•‘ƒ’’”‘ƒ…Šˆ‘”…ƒŽ…—Žƒ–‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡…‘˜‡”ƒ‰‡ˆ‘”•›–ƒš–‡•–‹‰Ǥ

B.2.5 Combinatorial Test Design Techniques

B.2.5.1 Introduction

Š‡ ƒ‹ ‘ˆ …‘„‹ƒ–‘”‹ƒŽ –‡•–‹‰ ‹• –‘ ”‡†—…‡ –Š‡ …‘•– ‘ˆ –‡•–‹‰ „› †‡”‹˜‹‰ ƒ •ƒŽŽ ȋ’‘••‹„Ž›
minimal) number of test cases that cover the chosen set of parameters and input values of the test
‹–‡Ǥ ‘„‹ƒ–‘”‹ƒŽ –‡•– †‡•‹‰ –‡…Š‹“—‡• ’”‘˜‹†‡ –Š‡ ƒ„‹Ž‹–› –‘ †‡”‹˜‡ –‡•– …ƒ•‡• ˆ”‘ ‹’—– ˜ƒŽ—‡•
–Šƒ– Šƒ˜‡ ’”‡˜‹‘—•Ž› „‡‡ •‡Ž‡…–‡†ǡ •—…Š ƒ• –Š”‘—‰Š –Š‡ ƒ’’Ž‹…ƒ–‹‘ ‘ˆ ‘–Š‡” •’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† –‡•–
†‡•‹‰ –‡…Š‹“—‡• Ž‹‡ ‡“—‹˜ƒŽ‡…‡ ’ƒ”–‹–‹‘‹‰ ‘” „‘—†ƒ”› ˜ƒŽ—‡ ƒƒŽ›•‹•Ǥ ƒ…Š –‡…Š‹“—‡ ™‹ŽŽ „‡
demonstrated through the application of one example. Since each technique shares common steps in
‹†‡–‹ˆ›‹‰ˆ‡ƒ–—”‡•‡–•ƒ††‡”‹˜‹‰–‡•–…‘†‹–‹‘•ǡ–Š‡•‡•–‡’•ƒ”‡†‡‘•–”ƒ–‡†‘…‡„‡Ž‘™ˆ‘”ƒŽŽ
–‡…Š‹“—‡•ǡƒ†–Š‹•‹•–Š‡ˆ‘ŽŽ‘™‡†„›–Š‡•–‡’•‘ˆ†‡”‹˜‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ†–‡•–…ƒ•‡•–Šƒ–ƒ”‡
unique to each combinatorial technique.

ǤʹǤͷǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the test basis for a test item travel_preference, which records the travel preferences of
•–ƒˆˆ ‡„‡”• ‘ˆ ƒ ‘”‰ƒ‹•ƒ–‹‘ –Šƒ– –”ƒ˜‡Ž –‘ ƒŒ‘” …ƒ’‹–ƒŽ …‹–‹‡• ˆ‘” ™‘” ’—”’‘•‡•Ǥ ƒ…Š •‡–
of travel preferences is chosen through three sets of radio buttons, which consist of the following
input value choices:

Destination = Paris, London, Sydney

Class = First, Business, Economy

Seat = Aisle, Window

ˆƒ˜ƒŽ‹†‹’—–…‘„‹ƒ–‹‘‹•’”‘˜‹†‡†–‘–Š‡’”‘‰”ƒ‹–™‹ŽŽ‘—–’—–Dz……‡’–dzǡ‘–Š‡”™‹•‡‹–™‹ŽŽ‘—–’—–Dz‡Œ‡…–dzǤ

B.2.5.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡––‘„‡†‡ϐ‹‡†ǣ
ͳǣ–”ƒ˜‡Ž̴’”‡ˆ‡”‡…‡ˆ—…–‹‘

B.2.5.4 Step 2: Derive Test Conditions (TD2)

˜‡”› …‘„‹ƒ–‘”‹ƒŽ –‡…Š‹“—‡ •Šƒ”‡• ƒ …‘‘ ƒ’’”‘ƒ…Š –‘ †‡”‹˜‹‰ –‡•– …‘†‹–‹‘•Ǥ Šƒ– ‹•ǡ –‡•–
…‘†‹–‹‘•…‘””‡•’‘†–‘‡ƒ…Š’ƒ”ƒ‡–‡”ȋȌ‘ˆ–Š‡–‡•–‹–‡–ƒ‹‰‘‘‡•’‡…‹ϐ‹…˜ƒŽ—‡ȋȌǡ”‡•—Ž–‹‰
in one P-V pair. This is repeated until all parameters are paired with their corresponding values. For the
example above, this results in the following P-V pairs:

TCOND1: Destination – Paris (for FS1)

68 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOND2: Destination – London (for FS1)

TCOND3: ‡•–‹ƒ–‹‘Ȃ›†‡› (for FS1)

TCOND4: Class – First (for FS1)

TCOND5: Class – Business (for FS1)

TCOND6: Žƒ••Ȃ…‘‘› (for FS1)

TCOND7: Seat – Aisle (for FS1)

TCOND8: Seat – Window (for FS1)

B.2.5.5 All Combinations

B.2.5.5.1 Step 3: Derive Test Coverage Items (TD3)

In all combinations testing, the test coverage items are the unique combinations of P-V pairs, made up
‘ˆ‘‡Ǧ’ƒ‹”ˆ‘”‡ƒ…Š–‡•–‹–‡’ƒ”ƒ‡–‡”ǤŠ‡•‡Ǧ’ƒ‹”•™‡”‡‡ƒ”Ž‹‡”‹†‡–‹ϐ‹‡†ƒ•–‡•–…‘†‹–‹‘•Ǥ

TCOVER1: Destination – Paris, Class – First, Seat – Aisle (for TCOND 1, 4, 7)

TCOVER2: Destination – Paris, Class – First, Seat – Window (for TCOND 1, 4, 8)

TCOVER3: Destination – Paris, Class – Business, Seat – Aisle (for TCOND 1, 5, 7)

TCOVER4: Destination – Paris, Class – Business, Seat – Window (for TCOND 1, 5, 8)

TCOVER5: Destination – Paris, Žƒ••Ȃ…‘‘›ǡ Seat – Aisle (for TCOND 1, 6, 7)

TCOVER6: Destination – Paris, Žƒ••Ȃ…‘‘›ǡ Seat – Window (for TCOND 1, 6, 8)

TCOVER7: Destination – London, Class – First, Seat – Aisle (for TCOND 2, 4, 7)

TCOVER8: Destination – London, Class – First, Seat – Window (for TCOND 2, 4, 8)

TCOVER9: Destination – London, Class – Business, Seat – Aisle (for TCOND 2, 5, 7)

TCOVER10: Destination – London, Class – Business, Seat – Window (for TCOND 2, 5, 8)

TCOVER11: Destination – London, Žƒ••Ȃ…‘‘›ǡ Seat – Aisle (for TCOND 2, 6, 7)

TCOVER12: Destination – London, Žƒ••Ȃ…‘‘›ǡ Seat – Window (for TCOND 2, 6, 8)

TCOVER13: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Class – First, Seat – Aisle (for TCOND 3, 4, 7)

TCOVER14: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Class – First, Seat – Window (for TCOND 3, 4, 8)

TCOVER15: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Class – Business, Seat – Aisle (for TCOND 3, 5, 7)

TCOVER16: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Class – Business, Seat – Window (for TCOND 3, 5, 8)

TCOVER17: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Žƒ••Ȃ…‘‘›ǡ Seat – Aisle (for TCOND 3, 6, 7)

TCOVER18: ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Žƒ••Ȃ…‘‘›ǡ Seat – Window (for TCOND 3, 6, 8)

B.2.5.5.2 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰‘‡Ǧ’ƒ‹”ƒ†…‘„‹‹‰‹–™‹–Š‡˜‡”›‘–Š‡”Ǧ’ƒ‹”ˆ”‘ƒŽŽ‘–Š‡”
’ƒ”ƒ‡–‡”•ȋ™Š‡”‡‡ƒ…Š…‘„‹ƒ–‹‘…”‡ƒ–‡•‡šƒ…–Ž›‘‡–‡•–…ƒ•‡Ȍǡ‹†‡–‹ˆ›‹‰ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•
© ISO/IEC 2015 – All rights reserved 69
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

–‘ ‡š‡”…‹•‡ ƒ› ‘–Š‡” ‹’—– ˜ƒ”‹ƒ„Ž‡ ”‡“—‹”‡† „› –Š‡ –‡•– …ƒ•‡ǡ †‡–‡”‹‹‰ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ƒ†
repeating until the required coverage is achieved. In this example, this results in the following test cases:

Table B.22 — Test cases for all combinations testing

Input Values
Expected Test Coverage Item
Test Case # Destination Class Seat Result
1 Paris First Aisle Accept TCOVER1
2 Paris First Window Accept TCOVER2
3 Paris Business Aisle Accept TCOVER3
4 Paris Business Window Accept TCOVER4
5 Paris …‘‘› Aisle Accept TCOVER5
6 Paris …‘‘› Window Accept TCOVER6
7 London First Aisle Accept TCOVER7
8 London First Window Accept TCOVER8
9 London Business Aisle Accept TCOVER9
10 London Business Window Accept TCOVER10
11 London …‘‘› Aisle Accept TCOVER11
12 London …‘‘› Window Accept TCOVER12
13 ›†‡› First Aisle Accept TCOVER13
14 ›†‡› First Window Accept TCOVER14
15 ›†‡› Business Aisle Accept TCOVER15
16 ›†‡› Business Window Accept TCOVER16
17 ›†‡› …‘‘› Aisle Accept TCOVER17
18 ›†‡› …‘‘› Window Accept TCOVER18

B.2.5.5.3 Step 5: Assemble Test Sets (TD5)

–ƒ›„‡†‡…‹†‡†–Šƒ–ƒŽŽ–‡•–…ƒ•‡•™‹ŽŽ„‡†‹˜‹†‡†‹–‘–Š‘•‡–Šƒ–…‘˜‡”ƒ‹•Ž‡•‡ƒ–•ƒ†™‹†‘™•‡ƒ–•Ǥ
This would result in the following test sets.

TS1: TEST CASES 1, 3, 5, 7, 9, 11, 13, 15, 17

TS2: TEST CASES 2, 4, 6, 8, 10, 12, 14, 16, 18

B.2.5.5.4 Step 6: Derive Test Procedures (TD6)

‹…‡‡ƒ…Š–‡•–•‡–‹•‰‘‹‰–‘„‡‡š‡…—–‡†„›ƒ†‹ˆˆ‡”‡––‡•–‡”ǡ–Š‡›…ƒ„‡†‹˜‹†‡†‹–‘–™‘–‡•–’”‘…‡†—”‡•Ǥ

ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

ʹǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.5.5.5 All Combinations Testing Coverage

Using the formula provided in 6.2.5.1 and the test coverage items derived above:
18
Coverage(all −combinations ) = × 100% = 100%
18
Thus, 100% coverage of test coverage items for all-combinations testing has been achieved.

70 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

B.2.5.6 Pair-wise Testing

B.2.5.6.1 Step 3: Derive Test Coverage Items (TD3)

 ’ƒ‹”Ǧ™‹•‡ –‡•–‹‰ǡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ ‹†‡–‹ϐ‹‡† ƒ• –Š‡ —‹“—‡ ’ƒ‹”• ‘ˆ Ǧ ’ƒ‹”• ˆ‘” †‹ˆˆ‡”‡–
parameters. For the travel_preference‡šƒ’Ž‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡†‡ϐ‹‡†ǣ

TCOVER1: Paris, First (for TCOND1, TCOND4)

TCOVER2: Paris, Business (for TCOND1, TCOND5)

TCOVER3: ƒ”‹•ǡ…‘‘› (for TCOND1, TCOND6)

TCOVER4: London, First (for TCOND2, TCOND4)

TCOVER5: London, Business (for TCOND2, TCOND5)

TCOVER6: ‘†‘ǡ…‘‘› (for TCOND2, TCOND6)

TCOVER7: ›†‡›ǡ ‹”•– (for TCOND3, TCOND4)

TCOVER8: ›†‡›ǡ—•‹‡•• (for TCOND3, TCOND5)

TCOVER9: ›†‡›ǡ…‘‘› (for TCOND3, TCOND6)

TCOVER10: Paris, Aisle (for TCOND1, TCOND7)

TCOVER11: Paris, Window (for TCOND1, TCOND8)

TCOVER12: London, Aisle (for TCOND2, TCOND7)

TCOVER13: London, Window (for TCOND2, TCOND8)

TCOVER14: ›†‡›ǡ‹•Ž‡ (for TCOND3, TCOND7)

TCOVER15: ›†‡›ǡ‹†‘™ (for TCOND3, TCOND8)

TCOVER16: First, Aisle (for TCOND4, TCOND7)

TCOVER17: First, Window (for TCOND4, TCOND8)

TCOVER18: Business, Aisle (for TCOND5, TCOND7)

TCOVER19: Business, Window (for TCOND5, TCOND8)

TCOVER20: …‘‘›ǡ‹•Ž‡ (for TCOND6, TCOND7)

TCOVER21: …‘‘›ǡ‹†‘™ (for TCOND6, TCOND8)

B.2.5.6.2 Step 4: Derive Test Cases (TD4)

‡•– …ƒ•‡• ƒ”‡ †‡”‹˜‡† „› •‡Ž‡…–‹‰ ‘‡ ‘” ‘”‡ —‹“—‡ ’ƒ‹”• ‘ˆ Ǧ ’ƒ‹”• ȋ–‡•– …‘˜‡”ƒ‰‡ ‹–‡•Ȍ ˆ‘”
‹…Ž—•‹‘‹–Š‡…—””‡––‡•–…ƒ•‡ǡ•‡Ž‡…–‹‰ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡”‡“—‹”‡†
„› –Š‡ –‡•– …ƒ•‡ǡ †‡–‡”‹‹‰ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ‘ˆ –Š‡ –‡•– ƒ† ”‡’‡ƒ–‹‰ —–‹Ž ƒŽŽ Ǧ ’ƒ‹”• ™‹–Š
different parameters are included in at least one test case. In this example, three P-V pairs can be
included in all test cases.

© ISO/IEC 2015 – All rights reserved 71

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.23 — Test cases pair-wise testing

Input Values
Test Case Expected Test Coverage Items
# Destination Class Seat Result
1 Paris First Aisle Accept TCOVER1, TCOVER10, TCOVER16
2 Paris Business Window Accept TCOVER2, TCOVER11, TCOVER19
3 Paris …‘‘› Aisle Accept TCOVER3, TCOVER10, TCOVER20
4 London First Aisle Accept TCOVER4, TCOVER12, TCOVER16
5 London Business Window Accept TCOVER5, TCOVER13, TCOVER19
6 London …‘‘› Aisle Accept TCOVER6, TCOVER12, TCOVER20
7 ›†‡› First Window Accept TCOVER7, TCOVER15, TCOVER17
8 ›†‡› Business Aisle Accept TCOVER8, TCOVER14, TCOVER18
9 ›†‡› …‘‘› Window Accept TCOVER9, TCOVER15, TCOVER21

B.2.5.6.3 Step 5: Assemble Test Sets (TD5)

–ƒ›„‡†‡…‹†‡†•‹…‡–Š‡”‡ƒ”‡ƒ”‡Žƒ–‹˜‡Ž›•ƒŽŽ—„‡”‘ˆ–‡•–…ƒ•‡•ǡ–Šƒ––Š‡›…ƒ„‡…‘„‹‡†‹–‘
the one test set, as follows.
TS1: TEST CASES 1, 2, 3, 4, 5, 6, 7, 8, 9

B.2.5.6.4 Step 6: Derive Test Procedures (TD6)

‹…‡–Š‡”‡‹•‘Ž›‘‡–‡•–•‡–ǡ‹–…ƒ„‡…‘„‹‡†‹–‘–Š‡‘‡–‡•–’”‘…‡†—”‡Ǥ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.5.6.5 Pair-wise Testing Coverage

Using the formula provided in 6.2.5.2 and the test coverage items derived above:
21
Coverage( pairwise ) = × 100% = 100%
21
Thus, 100% coverage of test coverage items for pair-wise testing has been achieved.

B.2.5.7 Each Choice Testing

B.2.5.7.1 Step 3: Derive Test Coverage Items (TD3)

In each choice (or 1-wise) testing, the test coverage items are the set of P-V pairs. Thus, for the travel_
preference‡šƒ’Ž‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡†‡ϐ‹‡†ǣ

TCOVER1: Destination – Paris (for TCOND1)

TCOVER2: Destination – London (for TCOND2)

TCOVER3: ‡•–‹ƒ–‹‘Ȃ›†‡› (for TCOND3)

TCOVER4: Class – First (for TCOND4)

TCOVER5: Class – Business (for TCOND5)

TCOVER6: Žƒ••Ȃ…‘‘› (for TCOND6)

72 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOVER7: Seat – Aisle (for TCOND7)

TCOVER8: Seat – Window (for TCOND8)

B.2.5.7.2 Step 4: Derive Test Cases (TD4)

ƒ…Š…Š‘‹…‡–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰‘‡‘”‘”‡Ǧ’ƒ‹”•ˆ‘”‹…Ž—•‹‘‹–Š‡…—””‡––‡•–…ƒ•‡ǡ
•‡Ž‡…–‹‰ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡•”‡“—‹”‡†„›–Š‡–‡•–…ƒ•‡ǡ†‡–‡”‹‹‰–Š‡
expected result and repeating until all P-V pairs are included in at least one test case. For this example,
‘Ž›–Š”‡‡–‡•–…ƒ•‡•ƒ”‡”‡“—‹”‡†ǣ

Table B.24 — Test cases for each choice testing

Input Values
Test Case Expected Test Coverage Items
# Destination Class Seat Result
1 Paris First Aisle Accept TCOVER1, TCOVER4, TCOVER7
2 London Business Window Accept TCOVER2, TCOVER5, TCOVER8
3 ›†‡› …‘‘› Aisle Accept TCOVER3, TCOVER6, TCOVER7

Note that other test cases could be derived that would also achieve the required level of coverage.

B.2.5.7.3 Step 5: Assemble Test Sets (TD5)

‹…‡ƒ˜‡”›•ƒŽŽ—„‡”‘ˆ–‡•–…ƒ•‡•†‡”‹˜‡†‹–Š‹•‡šƒ’Ž‡ǡ‹–ƒ›„‡†‡…‹†‡†–Šƒ––Š‡›„‡…‘„‹‡†
into the one test set.
TS1: TEST CASES 1, 2, 3

B.2.5.7.4 Step 6: Derive Test Procedures (TD6)

Since all test cases are in the one test set, we can derive one test procedure.
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.5.7.5 Each Choice Testing Coverage

Using the formula provided in 6.2.5.3 and the test coverage items derived above:
8
Coverage(each _ choice ) = × 100% = 100%
8
Thus, 100% coverage of test coverage items for each choice testing has been achieved.

B.2.5.8 Base Choice Testing

B.2.5.8.1 Step 3: Derive Test Coverage Items (TD3)

‡•– …‘˜‡”ƒ‰‡ ‹–‡• ˆ‘” „ƒ•‡ …Š‘‹…‡ –‡•–‹‰ ƒ”‡ …Š‘•‡ „› •‡Ž‡…–‹‰ ƒ Dz„ƒ•‡ …Š‘‹…‡dz ˜ƒŽ—‡ ˆ‘” ‡ƒ…Š
’ƒ”ƒ‡–‡”Ǥ ‘”‡šƒ’Ž‡ǡ–Š‡„ƒ•‡…Š‘‹…‡…‘—Ž†„‡…Š‘•‡ˆ”‘–Š‡‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡ǡˆ”‘–Š‡ƒ‹
path in use case testing or from the test coverage items that are derived during equivalence partitioning.
–Š‹•‡šƒ’Ž‡ǡ–Š‡‘’‡”ƒ–‹‘ƒŽ’”‘ϐ‹Ž‡ƒ›‹†‹…ƒ–‡–Šƒ––Š‡ˆ‘ŽŽ‘™‹‰‹’—–˜ƒŽ—‡••Š‘—Ž†„‡…Š‘•‡ƒ•
the base choice:

TCOVER1: Destination – London, Žƒ••Ȃ…‘‘›ǡ Seat – Window (covers TCOND2,

͸ƬͺȌ

© ISO/IEC 2015 – All rights reserved 73

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Š‡”‡ƒ‹‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰ƒŽŽ”‡ƒ‹‹‰Ǧ’ƒ‹”•ǣ

TCOVER2 Destination – Paris, Žƒ••Ȃ…‘‘›ǡ Seat – Window (cover s TCOND1,

͸ƬͺȌ

TCOVER3 ‡•–‹ƒ–‹‘Ȃ›†‡›ǡ Žƒ••Ȃ…‘‘›ǡ Seat – Window (cover s TCOND3,

͸ƬͺȌ

TCOVER4 Destination – London, Class – First, Seat – Window (cover s TCOND2 ,

ͶƬͺȌ

TCOVER5 Destination – London, Class – Business, Seat – Window (cover s TCOND2 ,

ͷƬͺȌ

TCOVER6 Destination – London, Žƒ••Ȃ…‘‘›ǡ Seat – Aisle (cover s TCOND2 ,

͸Ƭ͹Ȍ

B.2.5.8.2 Step 4: Derive Test Cases (TD4)

„ƒ•‡Ǧ…Š‘‹…‡–‡•–…ƒ•‡…ƒ‘™„‡†‡”‹˜‡†„›…‘„‹‹‰–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ǣ
ƒ•‡Š‘‹…‡ǣ‘†‘ǡ…‘‘›ǡ‹†‘™
Š‹•‹••Š‘™ƒ•–Š‡ϐ‹”•––‡•–…ƒ•‡–Š‡–ƒ„Ž‡„‡Ž‘™ǤŠ‡”‡ƒ‹‹‰–‡•–…ƒ•‡•…ƒ‘™„‡†‡”‹˜‡†„›
substituting one P-V pair into the base-choice test case per test and repeating until all P-V pairs are
covered:

Table B.25 — Test cases for base choice testing

Input Values
Expected Test Coverage Item
Test Case # Destination Class Seat Result
1 London …‘‘› Window Accept TCOVER1
2 Paris …‘‘› Window Accept TCOVER2
3 ›†‡› …‘‘› Window Accept TCOVER3
4 London First Window Accept TCOVER4
5 London Business Window Accept TCOVER5
6 London …‘‘› Aisle Accept TCOVER6

B.2.5.8.3 Step 5: Assemble Test Sets (TD5)

‹…‡ƒ•ƒŽŽ—„‡”‘ˆ–‡•–…ƒ•‡•†‡”‹˜‡†‹–Š‹•‡šƒ’Ž‡ǡ‹–ƒ›„‡†‡…‹†‡†–Šƒ––Š‡›„‡…‘„‹‡†‹–‘
the one test set.
TS1: TEST CASES 1, 2, 3, 4, 5, 6

B.2.5.8.4 Step 6: Derive Test Procedures (TD6)

Since all test cases are in the one test set, we can derive one test procedure.
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

74 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

B.2.5.8.5 Base Choice Testing Coverage

Using the formula provided in 6.2.5.4 and the test coverage items derived above:
6
Coverage( base _ choice ) = × 100% = 100%
6
Thus, 100% coverage of test coverage items for base choice testing has been achieved.

B.2.6 Decision Table Testing

B.2.6.1 Introduction

The aim of decision table testing is to derive a set of test cases that cover the logical associations
„‡–™‡‡‹’—–•ƒ†‘—–’—–•ȋ™Š‹…Šƒ”‡”‡’”‡•‡–‡†ƒ•ƒ•‡”‹‡•‘ˆ…‘†‹–‹‘•ƒ†ƒ…–‹‘•Ȍƒ••‘…‹ƒ–‡†„›
decision rules according to the chosen level of condition and action coverage.

ǤʹǤ͸Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘

Take a cheque debit function whose inputs are debit amount, account type and current balance and
whose outputs are new balance and action code. Account typeƒ›„‡’‘•–ƒŽȋǮ’ǯȌ‘”…‘—–‡”ȋǮ…ǯȌǤŠ‡
action codeƒ›„‡ǮƬǯǡǮǯǡǮƬǯ‘”Ǯǯǡ…‘””‡•’‘†‹‰–‘Ǯ’”‘…‡••†‡„‹–ƒ†•‡†‘—–Ž‡––‡”ǯǡǮ’”‘…‡••
†‡„‹–‘Ž›ǯǡǮ•—•’‡†ƒ……‘—–ƒ†•‡†‘—–Ž‡––‡”ǯƒ†Ǯ•‡†‘—–Ž‡––‡”‘Ž›ǯ”‡•’‡…–‹˜‡Ž›ǤŠ‡ˆ—…–‹‘
has the following test basis:
ˆ–Š‡”‡ƒ”‡•—ˆϔ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡‹–Š‡ƒ……‘—–‘”–Š‡‡™„ƒŽƒ…‡™‘—Ž†„‡™‹–Š‹–Š‡ƒ—–Š‘”‹•‡†
overdraft limit then the debit is processed. If the new balance would exceed the authorised overdraft
limit then the debit is not processed and if it is a postal account it is suspended. Letters are sent out for
ƒŽŽ–”ƒ•ƒ…–‹‘•‘’‘•–ƒŽƒ……‘—–•ƒ†ˆ‘”‘Ǧ’‘•–ƒŽƒ……‘—–•‹ˆ–Š‡”‡ƒ”‡‹•—ˆϔ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡
(i.e. the account would no longer be in credit).

B.2.6.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡––‘„‡†‡ϐ‹‡†ǣ
FS1: cheque debit function

B.2.6.4 Step 2: Derive Test Conditions (TD2)

The test conditions are the conditions and actions that can be derived from the test basis.
The conditions (C) are:

TCOND1 (C1): New balance in credit (for FS1)

TCOND2 (C2): New balance overdraft, but within authorised limit (for FS1)

TCOND3 (C3): Account is postal (for FS1)

The actions (A) are:

TCOND4 (A1): Process debit (for FS1)

TCOND5 (A2): Suspend account (for FS1)

TCOND6 (A3): Send out letter (for FS1)

© ISO/IEC 2015 – All rights reserved 75

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

B.2.6.5 Step 3: Derive Test Coverage Items (TD3)

Š‡†‡…‹•‹‘–ƒ„Ž‡‡ƒ„Ž‡•‹†‡–‹ϐ‹…ƒ–‹‘‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ•†‡…‹•‹‘”—Ž‡•‹–Š‡†‡…‹•‹‘–ƒ„Ž‡Ǥ
ƒ…Š…‘Ž—‘ˆ–Š‡†‡…‹•‹‘–ƒ„Ž‡‹•ƒ†‡…‹•‹‘”—Ž‡Ǥ‡…‹•‹‘–ƒ„Ž‡•ƒ›„‡ƒŽ•‘„‡”‡’”‡•‡–‡†™‹–Š
–Š‡†‡…‹•‹‘”—Ž‡•‹”‘™•”ƒ–Š‡”–Šƒ…‘Ž—•ǤŠ‡–ƒ„Ž‡…‘’”‹•‡•–™‘’ƒ”–•Ǥ–Š‡ϐ‹”•–’ƒ”–‡ƒ…Š
†‡…‹•‹‘”—Ž‡‹•–ƒ„—Žƒ–‡†ƒ‰ƒ‹•––Š‡…‘†‹–‹‘•ǤǮǯ‹†‹…ƒ–‡•–Šƒ––Š‡…‘†‹–‹‘—•–„‡ˆ‘”
–Š‡†‡…‹•‹‘”—Ž‡–‘ƒ’’Ž›ƒ†ƒǮ ǯ‹†‹…ƒ–‡•–Šƒ––Š‡…‘†‹–‹‘—•–„‡ ˆ‘”–Š‡†‡…‹•‹‘”—Ž‡
–‘ƒ’’Ž›Ǥ–Š‡•‡…‘†’ƒ”–ǡ‡ƒ…Š†‡…‹•‹‘”—Ž‡‹•–ƒ„—Žƒ–‡†ƒ‰ƒ‹•––Š‡ƒ…–‹‘•ǤǮǯ‹†‹…ƒ–‡•–Šƒ––Š‡
ƒ…–‹‘™‹ŽŽ„‡’‡”ˆ‘”‡†ǢƒǮ ǯ‹†‹…ƒ–‡•–Šƒ––Š‡ƒ…–‹‘™‹ŽŽ‘–„‡’‡”ˆ‘”‡†Ǣƒƒ•–‡”‹•ȋȗȌ‹†‹…ƒ–‡•
–Šƒ––Š‡…‘„‹ƒ–‹‘‘ˆ…‘†‹–‹‘•‹•‹ˆ‡ƒ•‹„Ž‡ƒ†•‘‘ƒ…–‹‘•ƒ”‡†‡ϐ‹‡†ˆ‘”–Š‡†‡…‹•‹‘”—Ž‡Ǥ™‘
‘”‘”‡…‘Ž—•ƒ›„‡…‘„‹‡†‹ˆ–Š‡›…‘–ƒ‹ƒ‘‘Ž‡ƒ…‘†‹–‹‘–Šƒ–†‘‡•‘–ƒˆˆ‡…––Š‡‘—–…‘‡
regardless of its value.
Š‡‡šƒ’Ž‡Šƒ•–Š‡ˆ‘ŽŽ‘™‹‰†‡…‹•‹‘–ƒ„Ž‡ǡ™Š‹…Š‹†‡–‹ϐ‹‡•ͺ†‡…‹•‹‘”—Ž‡•ǡ͸‘ˆ™Š‹…Šƒ”‡ˆ‡ƒ•‹„Ž‡
ƒ†Š‡…‡”‡•—Ž–‹–Š‡†‡ϐ‹‹–‹‘‘ˆ͸–‡•–…‘˜‡”ƒ‰‡‹–‡•ǣ

Table B.26 — Decision table of the cheque debit function

Decision Rules: 1 2 3 4 5 6 7 8
C1: New balance in credit F F F F T T T T
C2: New balance overdraft, F F T T F F T T
but within authorised limit
C3: Account is postal F T F T F T F T
A1: Process debit F F T T T T ȗ ȗ
A2: Suspend account F T F F F F ȗ ȗ
A3: Send out letter T T T T F T ȗ ȗ

NOTE 1 Although “T” and “F” have been used in the above decision table to denote “True” and “False”, other
notations could be used (e.g. the words “true” and “false” could be used instead).

ʹ  –Š‡ –ƒ„Ž‡ ƒ„‘˜‡ǡ „‘–Š …‘†‹–‹‘• ƒ† ƒ…–‹‘• ƒ”‡ „‹ƒ”› ȋ ‘” Ȍ …‘†‹–‹‘•ǡ ™Š‹…Š ”‡•—Ž–• ‹
ƒ DzŽ‹‹–‡†Ǧ‡–”›dz †‡…‹•‹‘ –ƒ„Ž‡Ǥ  Dz‡š–‡†‡† ‡–”›dz †‡…‹•‹‘ –ƒ„Ž‡•ǡ …‘†‹–‹‘• ƒ†Ȁ‘” ƒ…–‹‘• …ƒ ƒ••—‡
multiple values.

B.2.6.6 Step 4: Derive Test Cases (TD4)

‡•– …ƒ•‡• ƒ”‡ †‡”‹˜‡† „› •‡Ž‡…–‹‰ ‘‡ ‘” ‘”‡ ˆ‡ƒ•‹„Ž‡ †‡…‹•‹‘ ”—Ž‡• ˆ”‘ –Š‡ †‡…‹•‹‘ –ƒ„Ž‡ ƒ– ƒ
–‹‡ –Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ …‘˜‡”‡† „› ƒ –‡•– …ƒ•‡ǡ ‹†‡–‹ˆ›‹‰ ‹’—–• –‘ ‡š‡”…‹•‡ –Š‡ …‘†‹–‹‘ȋ•Ȍ
ƒ† ƒ…–‹‘•ȋ•Ȍ ‘ˆ –Š‡ †‡…‹•‹‘ ”—Ž‡ ƒ† ƒ”„‹–”ƒ”› ˜ƒŽ‹† ˜ƒŽ—‡• ˆ‘” ƒ› ‘–Š‡” ‹’—– ˜ƒ”‹ƒ„Ž‡• ”‡“—‹”‡†
„›–Š‡–‡•–…ƒ•‡ǡ†‡–‡”‹‹‰–Š‡‡š’‡…–‡†”‡•—Ž–ƒ†”‡’‡ƒ–‹‰–Š‡•‡•–‡’•—–‹Ž–Š‡”‡“—‹”‡†Ž‡˜‡Ž‘ˆ
test coverage is achieved. The following test cases would be required to achieve 100% decision table
coverage, and correspond to the decision rules in the decision table above (no test cases are derived for
†‡…‹•‹‘”—Ž‡•͹ƒ†ͺ•‹…‡–Š‡›ƒ”‡‹ˆ‡ƒ•‹„Ž‡Ȍǣ

Table B.27 — Test case table of the cheque debit function

CAUSES/INPUTS EFFECTS/RESULTS
account overdraft current debit New action
Test Coverage
Test Case –›’‡ limit balance amount Balance code Item
1 Ǯ…ǯ £100 -£70 £50 -£70 Ǯǯ 1
2 Ǯ’ǯ £1500 £420 £2000 £420 ǮƬǯ 2
3 Ǯ…ǯ £250 £650 £800 -£150 ǮƬǯ 3
4 Ǯ’ǯ £750 -£500 £200 -£700 ǮƬǯ 4
5 Ǯ…ǯ £1000 £2100 £1200 £900 Ǯǯ 5

76 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.27 (continued)

CAUSES/INPUTS EFFECTS/RESULTS
account overdraft current debit New action
Test Coverage
Test Case –›’‡ limit balance amount Balance code Item
6 Ǯ’ǯ £500 £250 £150 £100 ǮƬǯ 6

B.2.6.7 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡ƒ”‡‘Ž›•‹š–‡•–…ƒ•‡•”‡“—‹”‡†–‘…‘˜‡”ƒŽŽ†‡…‹•‹‘”—Ž‡•ǡ‹–ƒ›„‡†‡…‹†‡†–Šƒ–ƒŽŽ–‡•–
cases will be manual and will all be placed in the one test set.
TS1: TEST CASE 1, 2, 3, 4, 5, 6

B.2.6.8 Step 6: Derive Test Procedures (TD6)

Since all test cases are in the one test set, we can derive one test procedure.
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.6.9 Decision Table Testing Coverage

Using the formula provided in 6.2.6 and the test coverage items derived above:
6
Coverage(decision _ table _ testing ) = × 100% = 100%
6
Thus, 100% coverage of test coverage items for decision table testing has been achieved.

B.2.7 Cause-Effect Graphing

B.2.7.1 Introduction

The aim of cause-effect graphing is to derive test cases that cover the logical relationships between
causes (e.g. inputs) and effects (e.g. outputs) of a test item according to a chosen level of coverage.
The technique utilises a notation that allows a cause-effect graph of the test item to be designed that
illustrates relationships between causes and effects as well as explicit constraints placed on causes and
‡ˆˆ‡…–•ǤŠ‹•†‹ˆˆ‡”•ˆ”‘†‡…‹•‹‘–ƒ„Ž‡–‡•–‹‰‹™Š‹…Š…‘•–”ƒ‹–•ƒ”‡‘–‡š’Ž‹…‹–Ž›•–ƒ–‡†Ǥƒ–—”ƒŽŽ›ǡ
–Š‡–‡…Š‹“—‡‹•‘Ž›‡ˆˆ‡…–‹˜‡–‘–Š‡‡š–‡––Šƒ––Š‡‘†‡Ž…ƒ’–—”‡•–Š‡–‡•–„ƒ•‹•‘ˆ–Š‡–‡•–‹–‡Ǥ

ǤʹǤ͹Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘

Take a cheque debit function whose inputs are debit amount, account type and current balance and
whose outputs are new balance and action code. Account typeƒ›„‡’‘•–ƒŽȋǮ’ǯȌ‘”…‘—–‡”ȋǮ…ǯȌǤŠ‡
action codeƒ›„‡ǮƬǯǡǮǯǡǮƬǯ‘”Ǯǯǡ…‘””‡•’‘†‹‰–‘Ǯ’”‘…‡••†‡„‹–ƒ†•‡†‘—–Ž‡––‡”ǯǡǮ’”‘…‡••
†‡„‹–‘Ž›ǯǡǮ•—•’‡†ƒ……‘—–ƒ†•‡†‘—–Ž‡––‡”ǯƒ†Ǯ•‡†‘—–Ž‡––‡”‘Ž›ǯ”‡•’‡…–‹˜‡Ž›ǤŠ‡ˆ—…–‹‘
has the following test basis:
ˆ–Š‡”‡ƒ”‡•—ˆϔ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡‹–Š‡ƒ……‘—–‘”–Š‡‡™„ƒŽƒ…‡™‘—Ž†„‡™‹–Š‹–Š‡ƒ—–Š‘”‹•‡†
overdraft limit then the debit is processed. If the new balance would exceed the authorised overdraft
limit then the debit is not processed and if it is a postal account it is suspended. Letters are sent out for
ƒŽŽ–”ƒ•ƒ…–‹‘•‘’‘•–ƒŽƒ……‘—–•ƒ†ˆ‘”‘Ǧ’‘•–ƒŽƒ……‘—–•‹ˆ–Š‡”‡ƒ”‡‹•—ˆϔ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡
(i.e. the account would no longer be in credit).

B.2.7.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡––‘„‡†‡ϐ‹‡†ǣ

© ISO/IEC 2015 – All rights reserved 77

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

FS1: cheque debit function

B.2.7.4 Step 2: Derive Test Conditions (TD2)

The test conditions are the causes and effects that can be derived from the test basis.
The causes are:

TCOND1 (C1): New balance in credit (for FS1)

TCOND2 (C2): New balance overdraft, but within authorised limit (for FS1)

TCOND3 (C3): Account is postal (for FS1)

The effects are:

TCOND4 (A1): Process debit (for FS1)

TCOND5 (A2): Suspend account (for FS1)

TCOND6 (A3): Send out letter (for FS1)

A cause-effect graph shows the relationship between the causes and effects in a notation similar to that
—•‡†„›†‡•‹‰‡”•‘ˆŠƒ”†™ƒ”‡Ž‘‰‹……‹”…—‹–•ǤŠ‡–‡•–„ƒ•‹•‹•‘†‡ŽŽ‡†„›–Š‡‰”ƒ’Š•Š‘™„‡Ž‘™Ǥ

C1 A1

C2 ෺ A2

෻
C3 A3

Figure B.10 — Cause-effect graph of the cheque debit function (see below for notation)

ͳ Š‡Dz‡’–›dz‘†‡–Šƒ–…‘‡…–•ͳȀʹ–‘ͳȀʹȀ͵‹•ƒ…‘‡…–‘”‘†‡–Šƒ–‹•—•‡†–‘‰”‘—’–‘‰‡–Š‡”
two or more causes.

78 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Figure B.11 — Notation for illustrating relationships between causes and effects in cause-
effect graphing

NOTE 2 Although the following “constraint” notations are not required for the example demonstrated in
–Š‹•…Žƒ—•‡ǡ–Š‡›ƒ”‡‹…Ž—†‡†Š‡”‡ƒ•–Š‡›…‘ˆ‡”ƒ†˜ƒ–ƒ‰‡•‹‹†‡–‹ˆ›‹‰”‡“—‹”‡†ǡ’‡”‹––‡†ƒ†ˆ‘”„‹††‡
”‡Žƒ–‹‘•Š‹’•„‡–™‡‡…ƒ—•‡•ƒ†”‡Žƒ–‹‘•Š‹’•„‡–™‡‡‡ˆˆ‡…–•Ǥ—…Š…‘•–”ƒ‹–”‡Žƒ–‹‘•Š‹’•ƒ”‡‘–‡š’Ž‹…‹–Ž›
•–ƒ–‡†‹†‡…‹•‹‘–ƒ„Ž‡•ƒ†ƒ”‡‘ˆ–‡‹’Ž‹…‹–‹•’‡…‹ϐ‹…ƒ–‹‘•ǤŠ‡•‡‘–ƒ–‹‘•’”‘˜‹†‡ƒ‡ƒ•ˆ‘”˜‡”‹ˆ›‹‰
–Š‡‹–‡‰”‹–›‘ˆ–Š‡…ƒ—•‡Ǧ‡ˆˆ‡…–‰”ƒ’Šǡ–Š‡†‡…‹•‹‘–ƒ„Ž‡ƒ†–Š‡–‡•–…ƒ•‡•–Šƒ–ƒ”‡†‡”‹˜‡†ˆ”‘‹–Ǥ

© ISO/IEC 2015 – All rights reserved 79

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Figure B.12 — Notation for representing cause and effect constraints in cause-effect graphing

B.2.7.5 Step 3: Derive Test Coverage Items (TD3)

Š‡ …ƒ—•‡Ǧ‡ˆˆ‡…– ‰”ƒ’Š ‹• –Š‡ ”‡…ƒ•– ‹ –‡”• ‘ˆ ƒ †‡…‹•‹‘ –ƒ„Ž‡ ȋ‡Ǥ‰Ǥ •‡‡ ˆ‘” ‡šƒ’Ž‡ ȋ›‡”• ͳͻ͹ͻȌ
ƒ† ȋ—”•‹—Ž— ƒ† ”‘„‡”– ͳͻͻͷȌȌǡ ™Š‹…Š ‡ƒ„Ž‡• ‹†‡–‹ϐ‹…ƒ–‹‘ ‘ˆ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ȋ‹Ǥ‡Ǥ –Š‡
feasible decision rules in the decision table). Each column of the decision table is a decision rule. The
–ƒ„Ž‡ …‘’”‹•‡• –™‘ ’ƒ”–•Ǥ  –Š‡ ϐ‹”•– ’ƒ”– ‡ƒ…Š †‡…‹•‹‘ ”—Ž‡ ‹• –ƒ„—Žƒ–‡† ƒ‰ƒ‹•– –Š‡ …ƒ—•‡•Ǥ  Ǯǯ
‹†‹…ƒ–‡•–Šƒ––Š‡…ƒ—•‡—•–„‡ˆ‘”–Š‡†‡…‹•‹‘”—Ž‡–‘ƒ’’Ž›ƒ†ƒǮ ǯ‹†‹…ƒ–‡•–Šƒ––Š‡…ƒ—•‡
—•–„‡ ˆ‘”–Š‡†‡…‹•‹‘”—Ž‡–‘ƒ’’Ž›Ǥ–Š‡•‡…‘†’ƒ”–ǡ‡ƒ…Š†‡…‹•‹‘”—Ž‡‹•–ƒ„—Žƒ–‡†ƒ‰ƒ‹•–
–Š‡‡ˆˆ‡…–•ǤǮǯ‹†‹…ƒ–‡•–Šƒ––Š‡‡ˆˆ‡…–™‹ŽŽ‘……—”ǢƒǮ ǯ‹†‹…ƒ–‡•–Šƒ––Š‡‡ˆˆ‡…–™‹ŽŽ‘–‘……—”Ǣƒ
ƒ•–‡”‹•ȋȗȌ‹†‹…ƒ–‡•–Šƒ––Š‡…‘„‹ƒ–‹‘‘ˆ…ƒ—•‡•‹•‹ˆ‡ƒ•‹„Ž‡ƒ†•‘‘‡ˆˆ‡…–•ƒ”‡†‡ϐ‹‡†ˆ‘”–Š‡
decision rule.
Š‡‡šƒ’Ž‡Šƒ•–Š‡ˆ‘ŽŽ‘™‹‰†‡…‹•‹‘–ƒ„Ž‡ǡ™Š‹…Š‹†‡–‹ϐ‹‡••‹š–‡•–…‘˜‡”ƒ‰‡‹–‡•ȋ†‡…‹•‹‘”—Ž‡•͹
ƒ†ͺƒ”‡‘––‡•–…‘˜‡”ƒ‰‡‹–‡••‹…‡–Š‡›ƒ”‡‹ˆ‡ƒ•‹„Ž‡Ȍǣ

80 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.28 — Decision table of the cheque debit function

Decision Rules: 1 2 3 4 5 6 7 8
C1: New balance in credit F F F F T T T T
C2: New balance overdraft, F F T T F F T T
but within authorised limit
C3: Account is postal F T F T F T F T
A1: Process debit F F T T T T ȗ ȗ
A2: Suspend account F T F F F F ȗ ȗ
A3: Send out letter T T T T F T ȗ ȗ

B.2.7.6 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰‘‡‘”‘”‡ˆ‡ƒ•‹„Ž‡†‡…‹•‹‘”—Ž‡•ˆ”‘–Š‡†‡…‹•‹‘–ƒ„Ž‡–Šƒ–Šƒ˜‡
‘–„‡‡‹…Ž—†‡†‹ƒ–‡•–…ƒ•‡ǡ‹†‡–‹ˆ›‹‰‹’—–•–‘‡š‡”…‹•‡–Š‡…ƒ—•‡•ȋ•Ȍƒ†‡ˆˆ‡…–•ȋ•Ȍ‘ˆ–Š‡†‡…‹•‹‘
”—Ž‡ƒ†ƒ”„‹–”ƒ”›˜ƒŽ‹†˜ƒŽ—‡•ˆ‘”ƒ›‘–Š‡”‹’—–˜ƒ”‹ƒ„Ž‡”‡“—‹”‡†„›–Š‡–‡•–…ƒ•‡ǡ†‡–‡”‹‹‰–Š‡
expected result of the test case, and repeating these steps until all feasible decision rules are covered.
The following test cases achieve 100% cause-effect coverage and correspond to the decision rules in
–Š‡†‡…‹•‹‘–ƒ„Ž‡ƒ„‘˜‡ȋ‘–‡•–…ƒ•‡•ƒ”‡‰‡‡”ƒ–‡†ˆ‘”†‡…‹•‹‘”—Ž‡•͹ƒ†ͺƒ•–Š‡›ƒ”‡‹ˆ‡ƒ•‹„Ž‡Ȍǣ

Table B.29 — Test case table of the cheque debit function

CAUSES/INPUTS EFFECTS/RESULTS
Test Coverage
account overdraft current debit New action
Item
Test Case –›’‡ limit balance amount balance code
1 Ǯ…ǯ £100 -£70 £50 -£70 Ǯǯ 1
2 Ǯ’ǯ £1500 £420 £2000 £420 ǮƬǯ 2
3 Ǯ…ǯ £250 £650 £800 -£150 ǮƬǯ 3
4 Ǯ’ǯ £750 -£500 £200 -£700 ǮƬǯ 4
5 Ǯ…ǯ £1000 £2100 £1200 £900 Ǯǯ 5
6 Ǯ’ǯ £500 £250 £150 £100 ǮƬǯ 6

B.2.7.7 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡ƒ”‡‘Ž›•‹š–‡•–…ƒ•‡•”‡“—‹”‡†–‘…‘˜‡”ƒŽŽ†‡…‹•‹‘”—Ž‡•ǡ‹–ƒ›„‡†‡…‹†‡†–Šƒ–ƒŽŽ–‡•–
cases will be manual and will all be placed in the one test set.
TS1: TEST CASE 1, 2, 3, 4, 5, 6

B.2.7.8 Step 6: Derive Test Procedures (TD6)

Since all test cases are in the one test set, we can derive one test procedure.
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.7.9 Cause-Effect Graphing Coverage

Using the formula provided in 6.2.7 and the test coverage items derived above:
6
Coverage(cause −effect − graphing ) = × 100% = 100%
6
Thus, 100% coverage of test coverage items for cause-effect graphing has been achieved.

© ISO/IEC 2015 – All rights reserved 81

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

B.2.8 State Transition Testing

B.2.8.1 Introduction

The aim of state transition testing is to derive a set of test cases that cover transitions and/or states of
–Š‡–‡•–‹–‡ƒ……‘”†‹‰–‘–Š‡…Š‘•‡Ž‡˜‡Ž‘ˆ…‘˜‡”ƒ‰‡ǤŠ‡–‡…Š‹“—‡‹•„ƒ•‡†—’‘ƒƒƒŽ›•‹•‘ˆ–Š‡
–‡•–„ƒ•‹•‘ˆ–Š‡–‡•–‹–‡–‘‘†‡Ž‹–•„‡Šƒ˜‹‘—”„›•–ƒ–‡–”ƒ•‹–‹‘•Ǥ

ǤʹǤͺǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider a test item, manage_display_changes, with the following test basis:

The test item responds to input requests to change an externally held display mode for a time display device.
The external display mode can be set to one of four values: two correspond to displaying either the time or
the date, and the other two correspond to modes used when altering either the time or date.
Š‡”‡ƒ”‡ˆ‘—”’‘••‹„Ž‡‹’—–”‡“—‡•–•ǣǮŠƒ‰‡‘†‡ǯǡǮ‡•‡–ǯǡǮ‹‡‡–ǯƒ†Ǯƒ–‡‡–ǯǤǮŠƒ‰‡‘†‡ǯ
input request shall cause the display mode to move between the ‘display time’ and ‘display date’ values. If the
†‹•’Žƒ›‘†‡‹••‡––‘Ǯ†‹•’Žƒ›–‹‡ǯ‘”Ǯ†‹•’Žƒ›†ƒ–‡ǯ–Š‡ƒǮ‡•‡–ǯ‹’—–”‡“—‡•–•ŠƒŽŽ…ƒ—•‡–Š‡†‹•’Žƒ›‘†‡
to be set to the corresponding ‘alter time’ or ‘alter date’ modes. The ‘Time Set’ input request shall cause the
display mode to return to ‘display time’ from ‘alter time’ while similarly the ‘Date Set’ input request shall
cause the display mode to return to ‘display date’ from ‘alter date’.

B.2.8.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡‡‡†•–‘„‡†‡ϐ‹‡†ǣ
ͳǣ ƒƒ‰‡̴†‹•’Žƒ›̴…Šƒ‰‡•

B.2.8.4 Step 2: Derive Test Conditions (TD2)

•–ƒ–‡‘†‡Ž‹•’”‘†—…‡†ƒ•–Š‡–‡•–…‘†‹–‹‘Ǥ–ƒ–‡–”ƒ•‹–‹‘†‹ƒ‰”ƒ•ȋȌƒ”‡…‘‘Ž›—•‡†
as state models and their notation is illustrated below. A STD consists of states, transition, events
and actions (see Figure B.13ȌǤ˜‡–•ƒ”‡ƒŽ™ƒ›•…ƒ—•‡†„›‹’—–Ǥ‹‹Žƒ”Ž›ǡƒ…–‹‘•ƒ”‡Ž‹‡Ž›–‘…ƒ—•‡
‘—–’—–Ǥ Š‡ ‘—–’—– ˆ”‘ ƒ ƒ…–‹‘ ƒ› „‡ ‡••‡–‹ƒŽ ‹ ‘”†‡” –‘ ‹†‡–‹ˆ› –Š‡ …—””‡– •–ƒ–‡ ‘ˆ –Š‡ –‡•–
‹–‡Ǥ–”ƒ•‹–‹‘‹•†‡–‡”‹‡†„›–Š‡…—””‡–•–ƒ–‡ƒ†ƒ‡˜‡–ƒ†‹•‘”ƒŽŽ›Žƒ„‡ŽŽ‡†•‹’Ž›™‹–Š
the event and action. As explained in 5.2.8.1ǡ‹•–ƒ–‡–”ƒ•‹–‹‘–‡•–‹‰–Š‡–‡•–…‘†‹–‹‘•ƒ›„‡ƒŽŽ
states of the state model, all transitions of the state model or the entire state model, depending on the
coverage requirements of testing.

Figure B.13 — Generic state model

82 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

The STD for the test item manage_display_changes is as follows (which in this example is the test
condition TCOND1):

‘reset’ (R)
alter time (AT)

DISPLAYING CHANGING
TIME (S1) TIME (S3)
‘time set’ (TS)
display time (T)

‘change ‘change
mode’ (CM) mode’ (CM)
display display
time (T) date (D)

‘reset’ (R)
alter date (AD)

DISPLAYING CHANGING
DATE (S2) DATE (S4)
‘date set’ (DS)

Figure B.14 — State transition diagram for manage_display_changes

B.2.8.5 Step 3: Derive Test Coverage Items - 0-Switch and “All Transitions” Testing (TD3)

Assuming the chosen level of coverage is “all transitions”, a state table can be drawn to represent all
˜ƒŽ‹†ƒ†‹˜ƒŽ‹†–”ƒ•‹–‹‘•ȋ–Š‡”‡“—‹”‡†–‡•–…‘˜‡”ƒ‰‡‹–‡•ȌǤ‘ƒ…Š‹‡˜‡ˆ—ŽŽͲǦ•™‹–…Š…‘˜‡”ƒ‰‡‘Ž›
the valid transitions need to be exercised.
Ž‹‹–ƒ–‹‘‘ˆͲǦ•™‹–…Š…‘˜‡”ƒ‰‡‹•–Šƒ––Š‡–‡•–•ƒ”‡†‡”‹˜‡†–‘‡š‡”…‹•‡‘Ž›–Š‡˜ƒŽ‹†–”ƒ•‹–‹‘•‹
the test item. A more thorough test of the test item will also attempt to cause invalid transitions to
‘……—”ȋDzƒŽŽ–”ƒ•‹–‹‘•dzȌǤŠ‡‘Ž›‡š’Ž‹…‹–Ž›•Š‘™•–Š‡˜ƒŽ‹†–”ƒ•‹–‹‘•ȋƒŽŽ–”ƒ•‹–‹‘•‘–•Š‘™
ƒ”‡ …‘•‹†‡”‡† ‹˜ƒŽ‹†ȌǤ ‡ ‡šƒ’Ž‡ ‘ˆ ƒ •–ƒ–‡ ‘†‡Ž –Šƒ– ‡š’Ž‹…‹–Ž› •Š‘™• „‘–Š ˜ƒŽ‹† ƒ† ‹˜ƒŽ‹†
transitions is a state table, while an alternative representation is a state transition diagram that
includes an “anomalous” state at which all invalid transitions terminate. One notation used for state
–ƒ„Ž‡•‹•„”‹‡ϐŽ›†‡•…”‹„‡†„‡Ž‘™ǣ

Table B.30 — State table notation

Input 1 Input 2 etc.

Start State 1 –”› –”› etc.
Start State 2 –”› –”› etc.
etc. etc. etc. etc.

™Š‡”‡–”›α ‹‹•Š–ƒ–‡Ȁ—–’—–‘”…–‹‘ˆ‘”–Š‡‰‹˜‡•–ƒ”–•–ƒ–‡ƒ†‹’—–Ǥ

© ISO/IEC 2015 – All rights reserved 83

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

The state table for manage_display_changes is shown below:

Table B.31 — State table for manage_display_changes

 R TS DS
S1 S2/D S3/AT S1/– S1/–
S2 S1/T S4/AD S2/– S2/–
S3 S3/– S3/– S1/T S3/–
S4 S4/– S4/– S4/– S2/D

› ‡–”› ™Š‡”‡ –Š‡ •–ƒ–‡ ”‡ƒ‹• –Š‡ •ƒ‡ and the action is shown as null (–) represents a null
–”ƒ•‹–‹‘ǡ™Š‡”‡ƒ›actual transition that can be induced will represent a failure. It is the testing of
–Š‡•‡—ŽŽ–”ƒ•‹–‹‘•–Šƒ–‹•‹‰‘”‡†„›–‡•–•‡–•†‡•‹‰‡†Œ—•––‘ƒ…Š‹‡˜‡…‘˜‡”ƒ‰‡‘ˆ˜ƒŽ‹†–‡•–…‘˜‡”ƒ‰‡
items (0-switch). Thus a more complete test set (“all transitions”) will test both possible transitions
ƒ†—ŽŽ–”ƒ•‹–‹‘•ǡ™Š‹…Š‡ƒ•–‡•–‹‰–Š‡”‡•’‘•‡‘ˆ–Š‡–‡•–‹–‡–‘ƒŽŽ‹’—–••’‡…‹ϐ‹‡†‹–Š‡–‡•–
„ƒ•‹•‹ƒŽŽ’‘••‹„Ž‡•–ƒ–‡•ǤŠ‡•–ƒ–‡–ƒ„Ž‡’”‘˜‹†‡•ƒ‹†‡ƒŽ‡ƒ•‘ˆ†‹”‡…–Ž›†‡”‹˜‹‰–‡•–…‘˜‡”ƒ‰‡
items to cover null transitions (for “all transition” coverage).
There are 16 entries in the table above representing each of the four possible inputs that can occur in
each of the four possible states, making 16 test coverage items for “all transitions” coverage, which can
be read from the state table as shown below:

Table B.32 — State table to test case table mapping for manage_display_changes

 R TS DS
S2/D S3/AT S1/– S1/–
S1
(TCOVER1) (TCOVER2) (TCOVER3) (TCOVER4)
S1/T S4/AD S2/– S2/–
S2
(TCOVER5) (TCOVER6) (TCOVER7) (TCOVER8)
S3/– S3/– S1/T S3/–
S3
(TCOVER9) (TCOVER10) (TCOVER11) (TCOVER12)
S4/– S4/– S4/– S2/D
S4
(TCOVER13) (TCOVER14) (TCOVER15) (TCOVER16)

Š—•ǡ–Š‡ˆ‘ŽŽ‘™‹‰ȋ˜ƒŽ‹†ƒ†‹˜ƒŽ‹†Ȍ–‡•–…‘˜‡”ƒ‰‡‹–‡•™‡”‡‹†‡–‹ϐ‹‡†ˆ”‘–Š‡•–ƒ–‡–ƒ„Ž‡ƒ„‘˜‡ˆ‘”
“all transitions” coverage:

TCOVER1: ͳ–‘ʹ™‹–Š‹’—– (for FS1, valid transition)

TCOVER2: S1 to S3 with input R (for FS1, valid transition)

TCOVER3: S1 to S1 with input TS (for FS1, invalid transition)

TCOVER4: S1 to S1 with input DS (for FS1, invalid transition)

TCOVER5: ʹ–‘ͳ™‹–Š‹’—– (for FS1, valid transition)

TCOVER6: S2 to S4 with input R (for FS1, valid transition)

TCOVER7: S2 to S2 with input TS (for FS1, invalid transition)

TCOVER8: S2 to S2 with input DS (for FS1, invalid transition)

TCOVER9: ͵–‘͵™‹–Š‹’—– (for FS1, invalid transition)

84 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOVER10: S3 to S3 with input R (for FS1, invalid transition)

TCOVER11: S3 to S1 with input TS (for FS1, valid transition)

TCOVER12: S3 to S3 with input DS (for FS1, invalid transition)

TCOVER13: Ͷ–‘Ͷ™‹–Š‹’—– (for FS1, invalid transition)

TCOVER14: S4 to S4 with input R (for FS1, invalid transition)

TCOVER15: S4 to S4 with input TS (for FS1, invalid transition)

TCOVER16: S4 to S2 with input DS (for FS1, valid transition)

B.2.8.6 Step 4: Derive Valid Test Cases (TD4)

B.2.8.6.1 Options

Test cases can now be derived to exercise each of the possible transitions (using the abbreviated STD
Žƒ„‡Ž•ȌǤ’—–ȋ•Ȍ–‘‡š‡”…‹•‡–Š‡–”ƒ•‹–‹‘ȋ•Ȍ–Šƒ–ƒ”‡–‘„‡…‘˜‡”‡†„›‡ƒ…Š–‡•–…ƒ•‡…ƒ„‡†‡–‡”‹‡†
ˆ”‘–Š‡ǡƒ•…ƒ–Š‡‡š’‡…–‡†”‡•—Ž–ǡ™Š‹…Š…ƒ„‡†‡–‡”‹‡†„›…‘„‹‹‰–Š‡‡š’‡…–‡†‘—–’—–
ƒ†ϐ‹ƒŽ•–ƒ–‡‘ˆ–Š‡–”ƒ•‹–‹‘‹–Š‡Ǥ‡•–…ƒ•‡•ƒ›„‡†‡”‹˜‡†–‘…‘˜‡”‘‡–‘n transitions per
test case, where n is the maximum number of transitions possible. For example, test cases could be
†‡”‹˜‡† ˆ‘” ͲǦ•™‹–…Š ‘” ͳǦ•™‹–…Š …‘˜‡”ƒ‰‡ ȋƒŽ–Š‘—‰Š ‹ ’”ƒ…–‹…‡ ‹– ™‘—Ž† ‘– „‡ ‡…‡••ƒ”› –‘ †‡”‹˜‡
„‘–ŠͲǦ•™‹–…Šƒ†ͳǦ•™‹–…Š–‡•–…ƒ•‡•ǡ–Š‹•‹•†‡‘•–”ƒ–‡†„‡Ž‘™•‹’Ž›–‘‡š’Žƒ‹–Š‡ƒ’’”‘ƒ…ŠȌǤ‡•–
cases can also be derived to cover invalid transitions. These three scenarios are demonstrated below.

B.2.8.6.2 Step 4a: Derive 0-Switch Test Cases (Valid Transitions)

Š‡ ˆ‘ŽŽ‘™‹‰ •‹š –‡•– …ƒ•‡• ’”‘˜‹†‡ ͲǦ•™‹–…Š –‡•– …‘˜‡”ƒ‰‡Ǥ ƒ…Š –‡•– …ƒ•‡ ‹• †‡”‹˜‡† „› •‡Ž‡…–‹‰ ƒ
–”ƒ•‹–‹‘ƒ†‹†‡–‹ˆ›‹‰–Š‡‹’—–•ǡ‡š’‡…–‡†‘—–’—–ƒ†ϐ‹ƒŽ•–ƒ–‡ˆ”‘–Š‡—–‹ŽƒŽŽ–”ƒ•‹–‹‘•
ƒ”‡…‘˜‡”‡†„›‘‡–‡•–…ƒ•‡Ǥ

Table B.33 — 0-switch test cases for manage_display_changes

Test Case 1 2 3 4 5 6
Start State S1 S1 S2 S2 S3 S4
Input  R  R TS DS
Expected Output D AT T AD T D
Finish State S2 S3 S1 S4 S1 S2
Test Coverage Item 1 2 5 6 11 16

NOTE A test procedure could be written for the six text cases in the table above that would allow them to be
‡š‡…—–‡†•‡“—‡–‹ƒŽŽ›•‘–Šƒ––Š‡Dz ‹‹•Š–ƒ–‡dzˆ‘”‘‡–‡•–…ƒ•‡‹•–Š‡•–ƒ”–•–ƒ–‡‘ˆ–Š‡‡š–ȋ‡Ǥ‰Ǥ‡š‡…—–‹‘‘”†‡”
5, 1, 4, 6, 3, 2). This is elaborated on in step 5.

Š‹•‹†‹…ƒ–‡•–Šƒ–ˆ‘”–‡•–…ƒ•‡ͳ–Š‡•–ƒ”–‹‰•–ƒ–‡‹•
ȋͳȌǡ–Š‡‹’—–‹•Ǯ…Šƒ‰‡‘†‡ǯ
ȋȌǡ–Š‡‡š’‡…–‡†‘—–’—–‹•Ǯ†‹•’Žƒ›†ƒ–‡ǯȋȌǡƒ†–Š‡ϐ‹‹•Š•–ƒ–‡‹•
ȋʹȌǤ
These six test cases exercise each of the “valid” transitions and so achieves 0-switch coverage (Cho
ͳͻͺ͹ȌǤ‡•–•™”‹––‡–‘ƒ…Š‹‡˜‡–Š‹•Ž‡˜‡Ž‘ˆ…‘˜‡”ƒ‰‡ƒ”‡Ž‹‹–‡†‹–Š‡‹”ƒ„‹Ž‹–›–‘†‡–‡…–•‘‡–›’‡•‘ˆ
ˆƒ—Ž–•„‡…ƒ—•‡ƒŽ–Š‘—‰Š–Š‡›™‹ŽŽ†‡–‡…––Š‡‘•–‘„˜‹‘—•‹…‘””‡…––”ƒ•‹–‹‘•ƒ†‘—–’—–•ǡ–Š‡›™‹ŽŽ
‘–†‡–‡…–‘”‡•—„–Ž‡ˆƒ—Ž–•–Šƒ–ƒ”‡‘Ž›†‡–‡…–ƒ„Ž‡–Š”‘—‰Š‡š‡”…‹•‹‰•‡“—‡…‡•‘ˆ–”ƒ•‹–‹‘•Ǥ

B.2.8.6.3 Step 4b: Derive Test Cases for Invalid Transitions

‡•–…ƒ•‡•–‘…‘˜‡”‹˜ƒŽ‹†–”ƒ•‹–‹‘•…ƒ‘™„‡†‡ϐ‹‡†ƒ•ˆ‘ŽŽ‘™•ǡ™Š‡”‡DzȂDz”‡’”‡•‡–•ƒ—ŽŽ–”ƒ•‹–‹‘ǣ

© ISO/IEC 2015 – All rights reserved 85

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.34 — invalid test cases for manage_display_changes

Test Case 7 8 9 10 11 12 13 14 15 16
Start State S1 S1 S2 S2 S3 S3 S3 S4 S4 S4
Input TS DS TS DS  R DS  R TS
Expected Output – – – – – – – – – –
Finish State S1 S1 S2 S2 S3 S3 S3 S4 S4 S4
Test Coverage Item 3 4 7 8 9 10 12 13 14 15

As the table above shows, test cases that cover invalid test coverage items should not cause transition
ƒ™ƒ› ˆ”‘ –Š‡ •–ƒ”–‹‰ •–ƒ–‡Ǥ Š‡ –‡•– …ƒ•‡• ‹ –Š‡ ƒ„‘˜‡ –™‘ –ƒ„Ž‡• …‘„‹‡† ™‹ŽŽ ƒ…Š‹‡˜‡ DzƒŽŽ
transitions” coverage.

B.2.8.6.4 Step 4c: Derive Test Coverage Items - 1-Switch Testing (TD3)

The following test coverage items can be derived from the STD to achieve 1-switch coverage:

TCOVER17: ͳ–‘ʹ–‘ͳ™‹–Š‹’—–•ƒ† (for FS1)

TCOVER18: ͳ–‘ʹ–‘Ͷ™‹–Š‹’—–•ƒ† (for FS1)

TCOVER19: S1 to S3 to S1 with inputs R and TS (for FS1)

TCOVER20: ͵–‘ͳ–‘ʹ™‹–Š‹’—–•ƒ† (for FS1)

TCOVER21 S3 to S1 to S3 with inputs TS and R (for FS1)

TCOVER22: ʹ–‘ͳ–‘ʹ™‹–Š‹’—–•ƒ† (for FS1)

TCOVER23: ʹ–‘ͳ–‘͵™‹–Š‹’—–•ƒ† (for FS1)

TCOVER24: S2 to S4 to S2 with inputs R and DS (for FS1)

TCOVER25: Ͷ–‘ʹ–‘ͳ™‹–Š‹’—–•ƒ† (for FS1)

TCOVER26: S4 to S2 to S4 with inputs DS and R (for FS1)

B.2.8.6.5 Step 4d: Derive 1-Switch Test Cases (TD4)

If the test coverage chosen in step TD3 was to cover all 1-switch transitions, then test cases could be
written to exercise all possible sequential pairs of transitions. In this example, there are ten, as follows:

Table B.35 — 1-switch test cases for manage_display_changes

Test Case 17 18 19 20 21 22 23 24 25 26
Start State S1 S1 S1 S3 S3 S2 S2 S2 S4 S4
Input   R TS TS   R DS DS
Expected Output D D AT T T T T AD D D
Next State S2 S2 S3 S1 S1 S1 S1 S4 S2 S2
Input  R TS  R  R DS  R
Expected Output T AD T D AT D AT D T AD
Finish State S1 S4 S1 S2 S3 S2 S3 S2 S1 S4
Test Coverage Item 17 18 19 20 21 22 23 24 25 26

86 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Š‹• ‹†‹…ƒ–‡• –Šƒ– –‡•– …ƒ•‡ ͳ͹ …‘’”‹•‡• –™‘ –”ƒ•‹–‹‘•Ǥ ‘” –Š‡ ϐ‹”•– –”ƒ•‹–‹‘ –Š‡ •–ƒ”–‹‰ •–ƒ–‡
‹•
ȋͳȌǡ–Š‡‹‹–‹ƒŽ‹’—–‹•Ǯ…Šƒ‰‡‘†‡ǯȋȌǡ–Š‡‹–‡”‡†‹ƒ–‡‡š’‡…–‡†‘—–’—–‹•
†‹•’Žƒ›†ƒ–‡ȋȌǡƒ†–Š‡‡š–•–ƒ–‡‹•
ȋʹȌǤ ‘”–Š‡•‡…‘†–”ƒ•‹–‹‘ǡ–Š‡•‡…‘†‹’—–
‹•Ǯ…Šƒ‰‡‘†‡ǯȋȌǡ–Š‡ϐ‹ƒŽ‡š’‡…–‡†‘—–’—–‹•†‹•’Žƒ›–‹‡ȋȌǡƒ†–Š‡ϐ‹‹•Š•–ƒ–‡‹•

 ȋͳȌǤ ‘–‡ –Šƒ– –Š‡ ‹–‡”‡†‹ƒ–‡ •–ƒ–‡•ǡ ƒ† –Š‡ ‹’—–• ƒ† ‘—–’—–• ˆ‘” ‡ƒ…Š –”ƒ•‹–‹‘ǡ ƒ”‡
‡š’Ž‹…‹–Ž›†‡ϐ‹‡†Ǥ
Longer sequences of transitions can be tested to achieve higher and higher levels of switch coverage,
dependent on the level of test thoroughness required.

B.2.8.7 Step 5: Assemble Test Sets (TD5)

ˆƒ›„‡†‡…‹†‡†–Šƒ–ƒŽŽͲǦ•™‹–…Š–‡•–…ƒ•‡•ȋ…‘˜‡”‹‰˜ƒŽ‹†–”ƒ•‹–‹‘•Ȍ™‹ŽŽ„‡ƒ••‡„Ž‡†‹–‘‘‡–‡•–
set, the “all transitions” test cases covering invalid transitions in another test set and all 1-switch test
cases into a third, as follows:

TS1: 0-switch test cases – TEST CASES 1, 2, 3, 4, 5, 6.

TS2: “all transitions” INVALID test cases – TEST CASES 7, 8, 9, 10, 11, 12, 13, 14, 15, 16.

TS3: 1-switch test cases – TEST CASES 17, 18, 19, 20, 21, 22, 23, 24, 25, 26.
NOTE 1 In some instances, it is possible to order individual test cases such that the “Finish State” for one test
…ƒ•‡‹•–Š‡•–ƒ”–‹‰•–ƒ–‡‘ˆ–Š‡‡š–ǤŠ‹•…ƒ‹’”‘˜‡‡ˆϐ‹…‹‡…›†—”‹‰–‡•–‡š‡…—–‹‘ǤŠ‡’‘••‹„‹Ž‹–›‘ˆ†‘‹‰
–Š‹•†‡’‡†•‘–Š‡•’‡…‹ϐ‹…•–ƒ–‡‘†‡Ž„‡‹‰–‡•–‡†Ǥ–Š‡‡šƒ’Ž‡ƒ„‘˜‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰‘”†‡”‹‰‘ˆ–‡•–…ƒ•‡•
™‘—Ž†ƒ…Š‹‡˜‡–Š‹•‘„Œ‡…–‹˜‡ǣͳǣͲ•™‹–…Š–‡•–…ƒ•‡•ȂͷǡͳǡͶǡ͸ǡ͵ǡʹǤ

ʹ ‹…‡–Š‡ͳǦ•™‹–…Š–‡•–…ƒ•‡•†‡ϐ‹‡†ƒ„‘˜‡…‘˜‡”ƒŽŽ’ƒ–Š•‹–Š‡ͲǦ•™‹–…Š–‡•–…ƒ•‡•ǡ‹–‹•—‡…‡••ƒ”›
–‘ †‡ϐ‹‡ –‡•– •‡–• ƒ† –‡•– ’”‘…‡†—”‡• ˆ‘” –Š‡ ͲǦ•™‹–…Š –‡•– …ƒ•‡•Ǥ ‘™‡˜‡”ǡ –Š‡› ƒ”‡ ‹…Ž—†‡† Š‡”‡ ˆ‘”
completeness.

B.2.8.8 Step 6: Derive Test Procedures (TD6)

‡–‡•–’”‘…‡†—”‡…‘—Ž†„‡†‡ϐ‹‡†–‘‡š‡…—–‡ƒŽŽ–‡•–…ƒ•‡•‹–Š‡‘”†‡”–Šƒ––Š‡›ƒ”‡†‡ϐ‹‡†‹•–‡’ͷǣ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡʹƒ†͵ǡ‹–Š‡‘”†‡”–Š‡›ƒ”‡†‡ϐ‹‡†‹–Š‡–‡•–•‡–•Ǥ

B.2.8.9 State Transition Testing Coverage

Using the formula provided in 6.2.8 and the test coverage items derived above:
6
Coverage(0− switch _ cov erage ) = × 100% = 100%
6
16
Coverage(all _ transitions _ cov erage ) = × 100% = 100%
16
10
Coverage(1− switch _ cov erage ) = × 100% = 100%
10
Thus, 100% coverage of test coverage items for 0-switch testing, 1-switch testing and all-transitions
testing has been achieved.

B.2.9 Scenario Testing

B.2.9.1 Introduction

The aim of scenario testing is to derive test cases that cover the scenarios of the test item according to
–Š‡…Š‘•‡Ž‡˜‡Ž‘ˆ…‘˜‡”ƒ‰‡Ǥ…‡ƒ”‹‘–‡•–‹‰‹•„ƒ•‡†—’‘ƒƒƒŽ›•‹•‘ˆ–Š‡–‡•–„ƒ•‹•–‘’”‘†—…‡ƒ
‘†‡Ž‘ˆ‹–•„‡Šƒ˜‹‘—”‹–‡”•‘ˆ•‡“—‡…‡•‘ˆƒ…–‹‘•–Šƒ–…‘•–‹–—–‡™‘”ϐŽ‘™•–Š”‘—‰Š–Š‡–‡•–‹–‡Ǥ
© ISO/IEC 2015 – All rights reserved 87
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Š‡”‡ƒ”‡–™‘–›’‡•‘ˆ•…‡ƒ”‹‘–‡•–‹‰†‡‘•–”ƒ–‡†„‡Ž‘™ǤŠ‡ϐ‹”•–‡šƒ’Ž‡‹•„ƒ•‡†‘ƒ‰‡‡”‹…
ˆ‘”‘ˆ–Š‡–‡…Š‹“—‡ǡ™Š‹Ž‡–Š‡•‡…‘†‹•ƒ•’‡…‹ϐ‹…‡šƒ’Ž‡„ƒ•‡†‘—•‡…ƒ•‡•Ǥ

B.2.9.2 Example 1

ǤʹǤͻǤʹǤͳ ’‡…‹ϐ‹…ƒ–‹‘

Consider a test item withdraw_cash –Šƒ– ˆ‘”• ’ƒ”– ‘ˆ –Š‡ •›•–‡ –Šƒ– †”‹˜‡• ƒ —–‘ƒ–‡† ‡ŽŽ‡”
ƒ…Š‹‡ȋȌǡƒ†™Š‹…ŠŠƒ•–Š‡ˆ‘ŽŽ‘™‹‰–‡•–„ƒ•‹•ǣ
The withdraw_cash function allows customers with bank accounts to withdraw funds from their account
via an ATM. A withdrawal can only be made by a user with an open bank account, a valid card and matching
pin, and a working ATM. After the withdrawal is complete, the account balance is debited by the withdrawn
amount, a receipt for the withdrawal is printed, and the ATM is available and ready for the next user.
Š‡ˆ‘ŽŽ‘™‹‰•…‡ƒ”‹‘•Šƒ˜‡„‡‡•’‡…‹ϔ‹‡†ƒ•„‡‹‰”‡“—‹”‡†„›–Š‡…—•–‘‡”ǣ
Typical Scenario
— Successful withdrawal of funds from account.
Alternative Scenarios
Withdrawal not approved, because:
— –Š‡—•‡”ǯ•„ƒ…ƒ”†‹•”‡Œ‡…–‡†ƒ•‹–‹•—”‡…‘‰‹•‡†„›–Š‡
— the user enters their PIN incorrectly up to 2 times
— the user enters their PIN incorrectly three times, with the ATM retaining the card
— the user selects deposit or transfer instead of withdrawal
— the user selects an incorrect account that does not exist on the entered card
— the withdrawal amount entered by the user is invalid
— –Š‡”‡‹•‹•—ˆϔ‹…‹‡–…ƒ•Š‹–Š‡
— the user enters a non-dispensable amount
— the user enters an amount that exceeds their daily allowance
— –Š‡”‡ƒ”‡‹•—ˆϔ‹…‹‡–ˆ—†•‹–Š‡—•‡”ǯ•„ƒƒ……‘—–
 ”‡ƒŽ‹–›ǡƒ††‹–‹‘ƒŽ•…‡ƒ”‹‘•ǡ™Š‹…Šƒ††”‡•••‹–—ƒ–‹‘••—…Šƒ•–Š‡—•‡”’”‡••‹‰…ƒ…‡Žƒ–ƒ›’‘‹–
in the process, are also possible.

B.2.9.2.2 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: withdraw_cash function

B.2.9.2.3 Step 2: Derive Test Conditions (TD2)

‘‡ƒ„Ž‡–Š‡‹†‡–‹ϐ‹…ƒ–‹‘‘ˆ–‡•–…‘†‹–‹‘•ǡƒ‘†‡Ž‘ˆ–Š‡–‡•–‹–‡—•–„‡’”‘†—…‡†–Šƒ–‹†‡–‹ϐ‹‡•
the scenarios (and the activities within them) present in each scenario. The example model below is a
ϐŽ‘™Ǧ‘ˆǦ‡˜‡–•†‹ƒ‰”ƒǤ–Š‹•‘–ƒ–‹‘ǡ–Š‡Dzƒ‹dz’ƒ–Š‹•”‡’”‡•‡–‡†ƒ•ƒ–Š‹…„Žƒ…Ž‹‡ǡ–Š‡•–ƒ”–
ƒ† ‡† ’‘‹–• ‘ˆ –Š‡ ™‘”ϐŽ‘™ ƒ”‡ Žƒ„‡ŽŽ‡†ǡ ƒ† ‡ƒ…Š ƒ…–‹‘ ‹• –ƒ‰‰‡† ™‹–Š ƒ —‹“—‡ ‹†‡–‹ϐ‹‡” –Šƒ–
†‡•‹‰ƒ–‡•‹–ƒ•ƒ—•‡”ȋȌ‘”•›•–‡ȋȌȋ‹Ǥ‡Ǥ–‡•–‹–‡Ȍƒ…–‹‘Ǥ

88 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

START

U1 S1.1 U3.1
System
U4
User U2 S2.1 Press Select
Inserts Accepts Input PIN PIN Correct S4.1
Withdraw Account
Card Card

S1.2 S4.2
System U3.2 Invalid
Rejects Card Press Account
Deposit or GO BACK
Transfer
S9 S2.2 S10
PIN Incorrect Invoke
< 3 tries Deposit or
Transfer
END
S2.3 S3
PIN Incorrect
Retain Card
= 3 tries

END

S4.1 U5 S5.1 S6 S7 S8
Enter S9
Valid Suitable Dispense Debit Dispense
Withdraw Eject card
Account amount cash balance Receipt
Amount

S5.2 U6
Invalid User
U5 removes
amount GO BACK
card
END
S5.3
Insufficient U5
ATM Funds GO BACK

S5.4
Non-
dispensable
GO BACK
U5
amount

S5.5
Exceed daily U5
allowance GO BACK

S5.6
Insufficient
Account U5
GO BACK
Funds

Figure B.15 — Flow of events diagram for withdraw_cash function

In scenario testing, the test conditions are the main and alternative scenarios that are to be covered
†—”‹‰–‡•–‹‰ȋ‹Ǥ‡Ǥ–Š‡›ƒ”‡–Š‡•‡“—‡…‡•‘ˆ—•‡”ƒ†•›•–‡‹–‡”ƒ…–‹‘•–Š”‘—‰Š–Š‡ϐŽ‘™‘ˆ‡˜‡–•
†‹ƒ‰”ƒ–Šƒ–…‘•–‹–—–‡•‘‡•…‡ƒ”‹‘ȌǤŠ‡”‡™‡”‡ͳͳ•…‡ƒ”‹‘•†‡•…”‹„‡†‹–Š‡•’‡…‹ϐ‹…ƒ–‹‘ǡ‹…Ž—†‹‰
one main and ten alternatives. These can be described as test conditions (covering FS1) as follows:

TCOND1: Successful withdrawal of funds (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.1,
S6, S7, S8, S9, U6)

ʹǣ•‡”ǯ•…ƒ”†‹•—”‡…‘‰‹•‡†„› (covers U1, S1.2, S9, U6)

͵ǣ•‡”‡–‡”•‹…‘””‡…–Ž›δ͵–‹‡• (covers U1, S1.1, U2, S2.2)

Ͷǣ•‡”‡–‡”•’‹‹…‘””‡…–Ž›͵–‹‡• (covers U1, S1.1, U2, S2.2, U2, S2.2, U2, S2.3, S3)

TCOND5: User selects deposit or transfer (covers U1, S1.1, U2, S2.1, U3.2, S10)

© ISO/IEC 2015 – All rights reserved 89

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

TCOND6: User selects incorrect account (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.2)

TCOND7: User enters invalid withdrawal amount(covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.2)

ͺǣ•—ˆϐ‹…‹‡–…ƒ•Š‹–Š‡ (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.3)

TCOND9: User enters non-dispensable amount (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.4)

ͳͲǣ•‡”‡–‡”•ƒ‘—–‡š…‡‡†‹‰†ƒ‹Ž› (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.5)
allowance

ͳͳǣ•—ˆϐ‹…‹‡–ˆ—†•‹—•‡”ǯ•ƒ……‘—– (covers U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.6)

B.2.9.2.4 Step 3: Derive Test Coverage Items (TD3)

•…‡ƒ”‹‘–‡•–‹‰ǡ–Š‡–‡•–…‘†‹–‹‘•ƒ”‡–Š‡–›’‹…ƒŽƒ†ƒŽ–‡”ƒ–‹˜‡•…‡ƒ”‹‘•ǡ™Š‹…Šƒ”‡–Š‡•ƒ‡ƒ•
the test coverage items.

ͳαͳ ͹α͹

ʹαʹ ͺαͺ

͵α͵ ͻαͻ

ͶαͶ ͳͲαͳͲ

ͷαͷ ͳͳαͳͳ

͸α͸

B.2.9.2.5 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰ƒ•…‡ƒ”‹‘–‘…‘˜‡”ǡ‹†‡–‹ˆ›‹‰‹’—–•–‘‡š‡”…‹•‡–Š‡’ƒ–Š…‘˜‡”‡†„›
the test case, determining the expected result of the test and repeating until all scenarios are covered
ƒ•”‡“—‹”‡†ǤŠ‡•–‡’•‘ˆ–Š‡–‡•–…ƒ•‡ƒ”‡–›’‹…ƒŽŽ›™‘”†‡†‹ƒ–—”ƒŽŽƒ‰—ƒ‰‡ˆ‘”ƒ–Ǥˆ™‡ƒ••—‡
–Šƒ–‘‡–‡•–…ƒ•‡‹•”‡“—‹”‡†–‘…‘˜‡”‡ƒ…Š–‡•–…‘˜‡”ƒ‰‡‹–‡–Šƒ–™ƒ•‹†‡–‹ϐ‹‡†ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–
cases could be derived.
ͳ ‹–Š‹‡ƒ…Š–‡•–…ƒ•‡ǡ–Š‡”‡ƒ”‡ƒ™‹†‡˜ƒ”‹‡–›‘ˆ‹’—–˜ƒŽ—‡•–Šƒ–…‘—Ž†„‡…Š‘•‡–‘’‘’—Žƒ–‡‡ƒ…Š
‹’—–ϐ‹‡Ž†Ǥ“—‹˜ƒŽ‡…‡ƒ”–‹–‹‘‹‰…ƒ„‡—•‡†–‘†‡”‹˜‡ƒ•‡–‘ˆ˜ƒŽ—‡•ˆ‘”’‘’—Žƒ–‹‰‡ƒ…Š‹’—–ϐ‹‡Ž†Ǥ

Table B.36 — Test cases for scenario testing

Test Case # 1
Test Case Name Successful withdrawal of funds
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.1, S6, S7, S8, S9, U6
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͷͲǡͲͲͲ
Customer Account Balance – $100
Withdrawal amount – $50
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†
…ƒ”†ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰

90 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.36 (continued)

Expected Result ‹–Š†”ƒ™ƒŽŠƒ••—……‡••ˆ—ŽŽ›„‡‡ƒ†‡ˆ”‘…—•–‘‡”ƒ……‘—–
„ƒŽƒ…‡‹•̈́ͶͻǡͻͷͲ
Customer account balance is $50
‹•‘’‡ǡ‘’‡”ƒ–‹‘ƒŽƒ†ƒ™ƒ‹–‹‰ƒ…—•–‘‡”…ƒ”†ƒ•‹’—–
Test Coverage Item TCOVER1

Table B.37 — Test cases for scenario testing

Test Case # 2
Test Case Name •‡”ǯ•…ƒ”†‹•—”‡…‘‰‹•‡†„›
Scenario Path Exercised U1, S1.2, S9, U6
Input Invalid card
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†
…ƒ”†ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ƒ”†‹•”‡Œ‡…–‡†„›–Š‡™‹–Šƒ‡””‘”‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡…ƒ”†‹•
‹˜ƒŽ‹†ǤŠ‡‹•™ƒ‹–‹‰ˆ‘”–Š‡…—•–‘‡”–‘”‡‘˜‡–Š‡‹˜ƒŽ‹†…ƒ”†Ǥ
Test Coverage Item TCOVER2

Table B.38 — Test cases for scenario testing

Test Case # 3
Test Case Name •‡”‡–‡”•‹…‘””‡…–Ž›δ͵–‹‡•
Scenario Path Exercised U1, S1.1, U2, S2,2
Input Valid card with valid customer account – assume 293910982246 is valid
Invalid PIN entered twice – assume 0000 is invalid and does not match card
ƒŽƒ…‡Ȃ̈́ͳͲͲ
Customer Account Balance – $500
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ‹•”‡Œ‡…–‡†„›–Š‡Ǥ›•–‡’”‘’–•—•‡”–‘”‡Ǧ‡–‡”–Š‡‹”’‹Ǥ
Test Coverage Item TCOVER3

Table B.39 — Test cases for scenario testing

Test Case # 4
Test Case Name •‡”‡–‡”•‹…‘””‡…–Ž›͵–‹‡•
Scenario Path Exercised U1, S1.1, U2, S2.2, U2, S2.2, U2, S2.3, S3
Input Valid card with valid customer account – assume 293910982246 is valid
Invalid PIN entered three times – assume 0000 is invalid and does not match card
ƒŽƒ…‡Ȃ̈́ͳͲͲ
Customer Account Balance – $500
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰

© ISO/IEC 2015 – All rights reserved 91

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.39 (continued)

Expected Result ƒ…Š–‹‡‹˜ƒŽ‹†‹•‡–‡”‡†ǡ•›•–‡†‹•’Žƒ›•ƒ‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ–‹•
Test Coverage Item
‹…‘””‡…–ƒ†’”‘’–•—•‡”–‘‡–‡”ƒ‰ƒ‹Ǥ–Š‡–Š‹”†–”›ǡ•›•–‡”‡–ƒ‹•–Š‡
…ƒ”†ƒ††‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ–…ƒ”†Šƒ•„‡‡”‡–ƒ‹‡†ƒ†–Šƒ–—•‡”
•Š‘—Ž†…‘–ƒ…––Š‡‹”„ƒ–‘”‡…‘˜‡”‹–Ǥ”‡–ƒ‹•–Š‡—•‡”ǯ•…ƒ”†Ǥ‹•‘’‡ǡ
operational and awaiting a customer card as input.
TCOVER4

Table B.40 — Test cases for scenario testing

Test Case # 5
Test Case Name User selects deposit or transfer
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.2, S10
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
”ƒ•ƒ…–‹‘›’‡Ȃ‡’‘•‹–
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡’”‘’–‹‰–Š‡—•‡”–‘‡–‡”†‡’‘•‹–†‡–ƒ‹Ž•
Test Coverage Item TCOVER5

Table B.41 — Test cases for scenario testing

Test Case # 6
Test Case Name User selects incorrect account
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.2
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͳͲͲ
Customer selects incorrect account that does not exist on the card
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡ƒ……‘—–‡–‡”‡†‹•‹˜ƒŽ‹†ǡƒ†
prompts the user to select a new account
Test Coverage Item TCOVER6

Table B.42 — Test cases for scenario testing

Test Case # 7
Test Case Name User enters invalid withdrawal amount
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.2
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͳͲͲ
Customer Account Balance – $20
Withdrawal amount – $17
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰

92 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.42 (continued)

Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡ƒ‘—–‡–‡”‡†‹•‹˜ƒŽ‹†ǡƒ†
prompts the user to enter a new amount
Test Coverage Item TCOVER7

Table B.43 — Test cases for scenario testing

Test Case # 8
Test Case Name •—ˆϐ‹…‹‡–…ƒ•Š‹–Š‡
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.3
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͳͲͲ
Customer Account Balance – $500
Withdrawal amount – $200
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†
…ƒ”†ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡”‡ƒ”‡‹•—ˆϐ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡‹
–Š‡ǡƒ†’”‘’–•–Š‡—•‡”–‘‡–‡”ƒ‡™ƒ‘—–
Test Coverage Item TCOVER8

Table B.44 — Test cases for scenario testing

Test Case # 9
Test Case Name User enters non-dispensable amount
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.4
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͳͲͲǡƒ†‘Ž›…‘–ƒ‹•̈́ͷͲ‘–‡•
Customer Account Balance – $1,000
Withdrawal amount – $20
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†
…ƒ”†ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ–†‡‘‹ƒ–‹‘‡–‡”‡†…ƒ‘–„‡†‹•-
’‡•‡„›–Š‡ǡƒ†’”‘’–•–Š‡—•‡”–‘‡–‡”ƒ‡™ƒ‘—–
Test Coverage Item TCOVER9

Table B.45 — Test cases for scenario testing

Test Case # 10
Test Case Name •‡”‡–‡”•ƒ‘—–‡š…‡‡†‹‰†ƒ‹Ž›ƒŽŽ‘™ƒ…‡
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.5

© ISO/IEC 2015 – All rights reserved 93

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.45 (continued)

Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͳͲͲǡƒ†‘Ž›…‘–ƒ‹•̈́ͷͲ‘–‡•
Customer Account Balance – $3,000
—•–‘‡”ƒš‹—ƒ‹Ž›ŽŽ‘™ƒ…‡Ȃ̈́ͳǡͲͲͲ
Withdrawal amount – $2,000
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†…ƒ”†
ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡ƒ‘—–‡–‡”‡†‡š…‡‡†•–Š‡—•‡”ǯ•
†ƒ‹Ž›ƒŽŽ‘™ƒ…‡ǡƒ†’”‘’–•–Š‡—•‡”–‘‡–‡”ƒ‡™ƒ‘—–
Test Coverage Item TCOVER10

Table B.46 — Test cases for scenario testing

Test Case # 11
Test Case Name •—ˆϐ‹…‹‡–ˆ—†•‹–Š‡—•‡”ǯ•ƒ……‘—–
Scenario Path Exercised U1, S1.1, U2, S2.1, U3.1, U4, S4.1, U5, S5.6
Input Valid card with valid customer account – assume 293910982246 is valid
Valid PIN – assume 5652 is valid and matches card
ƒŽƒ…‡Ȃ̈́ͷͲǡͲͲͲ
Customer Account Balance – $20
Withdrawal amount – $50
Pre-condition ™‹–Š†”ƒ™ƒŽ…ƒ‘Ž›„‡ƒ†‡„›ƒ—•‡”™‹–Šƒ‘’‡„ƒƒ……‘—–ǡƒ˜ƒŽ‹†
…ƒ”†ƒ†ƒ–…Š‹‰’‹ǡƒ†ƒ™‘”‹‰
Expected Result ›•–‡†‹•’Žƒ›•‡••ƒ‰‡‹†‹…ƒ–‹‰–Šƒ––Š‡”‡ƒ”‡‹•—ˆϐ‹…‹‡–ˆ—†•ƒ˜ƒ‹Žƒ„Ž‡
in user’s bank account, and prompts user to enter a new amount
Test Coverage Item TCOVER11

ʹ Š‡–‡•–…ƒ•‡•ƒ„‘˜‡ƒ”‡‡ƒ…Š…‘–ƒ‹‡†‹ƒ•‡’ƒ”ƒ–‡–ƒ„Ž‡ˆ‘””‡ƒ†ƒ„‹Ž‹–›Ǥ’”ƒ…–‹…‡ǡ–Š‡›…‘—Ž†„‡

contained in one table.

B.2.9.2.6 Step 5: Assemble Test Sets (TD5)

‡•–•…‘—Ž†„‡‰”‘—’‡†ƒ……‘”†‹‰–‘™Š‡–Š‡”–Š‡›…‘˜‡”–Š‡ƒ‹‘”ƒŽ–‡”ƒ–‹˜‡•…‡ƒ”‹‘•ǣ

TS1: TEST CASE 1.

TS2: TEST CASES 2, 3, 4, 5, 6, 7, 8, 9, 10, 11.

B.2.9.2.7 Step 6: Derive Test Procedures (TD6)

Two procedures are required as follows:

ͳǣ…‘˜‡”‹‰–Š‡–‡•–…ƒ•‡‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

ʹǣ…‘˜‡”‹‰–Š‡–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

94 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

B.2.9.2.8 Scenario Testing Coverage

Using the formula provided in 6.2.9 and the test coverage items derived above:
11
Coverage( scenario) = × 100% = 100%
11

B.2.9.3 Example 2

B.2.9.3.1 Introduction

Use case testing is a form of scenario testing, in which test case derivation is based on a use case model
of the test item. It is demonstrated here using a separate example to provide users of this standard with
ƒˆ—ŽŽ›™‘”‡†‡šƒ’Ž‡‘ˆ–Š‹•–‡…Š‹“—‡Ǥ

ǤʹǤͻǤ͵Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the following example use case for a test item change_password:

Table B.47 — Example use case for change_password

Use Case ID UC001

Use Case …Šƒ‰‡̴’ƒ••™‘”†
Purpose To allow a user to change their existing password to a new password
Actors User
Description This use case allows users to change their current password to a new password.
Trigger •‡”…Ž‹…•Šƒ‰‡ƒ••™‘”†„—––‘‘–Š‡ƒ‹‡—•…”‡‡
Preconditions •‡”—•–ƒŽ”‡ƒ†›„‡Ž‘‰‰‡†‹–‘–Š‡•›•–‡
Scenario Name Step Action
Basic Flow 1 User clicks Change Password button
2 ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
3 •‡”‡–‡”•–Š‡‹”…—””‡–’ƒ••™‘”†…‘””‡…–Ž›
4 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž›
5 •‡””‡Ǧ‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž›
6 User clicks OK
7 ›•–‡†‹•’Žƒ›•‡••ƒ‰‡Dzƒ••™‘”†…Šƒ‰‡†•—……‡••ˆ—ŽŽ›dz
Alternative Flow – 3.1 •‡”‡–‡”•–Š‡‹”…—””‡–’ƒ••™‘”†‹…‘””‡…–Ž›
Existing Password
3.2 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž›
Incorrect
3.3 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž›
3.4 User clicks OK
3.5 ›•–‡†‹•’Žƒ›•ƒ‡””‘”‡••ƒ‰‡Dz—””‡–’ƒ••™‘”†‡–‡”‡†‹…‘””‡…–-
Ž›ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdzƒ†Š‹‰ŠŽ‹‰Š–•ƒŽŽ–‡š–‹–Š‡—””‡–ƒ••™‘”†ϐ‹‡Ž†
Alternative Flow – New 4.1 User enters a new password that is less than 8 characters long
Password Less Than 8
4.2 User clicks OK
Characters
4.3 ›•–‡†‹•’Žƒ›•ƒ‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†—•–„‡ƒ–Ž‡ƒ•–ͺ
…Šƒ”ƒ…–‡”•Ž‘‰ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz

© ISO/IEC 2015 – All rights reserved 95

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.47 (continued)

Use Case ID UC001
Use Case …Šƒ‰‡̴’ƒ••™‘”†
Purpose To allow a user to change their existing password to a new password
Actors User
Description This use case allows users to change their current password to a new password.
Trigger •‡”…Ž‹…•Šƒ‰‡ƒ••™‘”†„—––‘‘–Š‡ƒ‹‡—•…”‡‡
Preconditions •‡”—•–ƒŽ”‡ƒ†›„‡Ž‘‰‰‡†‹–‘–Š‡•›•–‡
Scenario Name Step Action
Alternative Flow – New 5.1 User enters a new password that is the same as their current password
Password Same as Cur-
5.2 User clicks OK
rent Password
5.3 ›•–‡†‹•’Žƒ›•‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†—•–‘–„‡–Š‡•ƒ‡ƒ•
…—””‡–’ƒ••™‘”†ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz
Alternative Flow – New 6.1 User re-enters new password that does not match the new password
Passwords Do Not –Š‡›‡–‡”‡†ƒ–•–‡’Ͷ
ƒ–…Š
6.2 User clicks OK
6.3 ›•–‡†‹•’Žƒ›•‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†•†‘‘–ƒ–…ŠǤŽ‡ƒ•‡
–”›ƒ‰ƒ‹Ǥdz
Variants and Ex- None
ceptions
Rules New password must be different from current password
New password must be at least 8 characters long
›•–‡™‹ŽŽƒ•ƒŽŽ…—””‡–ƒ†‡™’ƒ••™‘”†…Šƒ”ƒ…–‡”•™‹–Šƒƒ•–‡”‹•ȋȗȌ
”‡“—‡…› •‡†–Š‡ϐ‹”•––‹‡ƒ‡™—•‡”Ž‘‰•‹–‘–Š‡•›•–‡
›’‹…ƒŽŽ›—•‡†–™‹…‡’‡”—•‡”’‡”›‡ƒ”‘ƒ˜‡”ƒ‰‡
ƒ„‡‹˜‘‡†ƒ›–‹‡–Š‡—•‡”…Ž‹…•–Š‡DzŠƒ‰‡ƒ••™‘”†dz„—––‘

NOTE Additional scenarios, which address situations such as the user entering invalid characters for their
new password, could also exist.

B.2.9.3.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: change_password function

B.2.9.3.4 Step 2: Derive Test Conditions (TD2)

‡•–…‘†‹–‹‘•ƒ›„‡–Š‡–›’‹…ƒŽƒ†ƒŽ–‡”ƒ–‹˜‡•…‡ƒ”‹‘•’”‡•‡–‹–Š‡—•‡…ƒ•‡Ǥ–Š‹•‡šƒ’Ž‡
–Š‡›ƒ”‡ǣ

TCOND1: ƒ‹ Ž‘™ (for FS1)

TCOND2: Alternative Flow – Existing Password Incorrect (for FS1)

TCOND3: Alternative Flow – New Password Less Than 8 Characters (for FS1)

TCOND4: Alternative Flow – New Password Same as Current Password (for FS1)

TCOND5: Ž–‡”ƒ–‹˜‡ Ž‘™Ȃ‡™ƒ••™‘”†•‘‘–ƒ–…Š (for FS1)

96 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

B.2.9.3.5 Step 3: Derive Test Coverage Items (TD3)

Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•‹—•‡…ƒ•‡–‡•–‹‰ƒ”‡–Š‡–›’‹…ƒŽƒ†ƒŽ–‡”ƒ–‹˜‡•…‡ƒ”‹‘•ǡƒ•ˆ‘ŽŽ‘™•Ǥ

TCOVER1: ƒ‹ Ž‘™ (for TCOND1)

TCOVER2: Alternative Flow – Existing Password Incorrect (for TCOND2)

TCOVER3: Alternative Flow – New Password Less Than 8 Characters (for TCOND3)

TCOVER4: Alternative Flow – New Password Same as Current Password (for TCOND4)

TCOVER5: Ž–‡”ƒ–‹˜‡ Ž‘™Ȃ‡™ƒ••™‘”†•‘‘–ƒ–…Š (for TCOND5)

B.2.9.3.6 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›•‡Ž‡…–‹‰ƒ•…‡ƒ”‹‘–‘…‘˜‡”ǡ‹†‡–‹ˆ›‹‰‹’—–•–‘‡š‡”…‹•‡–Š‡’ƒ–Š…‘˜‡”‡†
„› –Š‡ –‡•– …ƒ•‡ǡ †‡–‡”‹‹‰ –Š‡ ‡š’‡…–‡† ”‡•—Ž– ƒ† ”‡’‡ƒ–‹‰ —–‹Ž ƒŽŽ —•‡ …ƒ•‡ •…‡ƒ”‹‘• ƒ”‡
covered as required.

Table B.48 — Test cases for use case testing

Use Case Name …Šƒ‰‡̴’ƒ••™‘”†

Test Case Name ƒ‹ Ž‘™
Description •‡”•—……‡••ˆ—ŽŽ›…Šƒ‰‡•–Š‡‹”’ƒ••™‘”†
Actors User
Test Coverage Item TCOVER1
Use Case Steps Covered 1, 2, 3, 4, 5, 6, 7
Preconditions •‡”‹•ƒŽ”‡ƒ†›Ž‘‰‰‡†‹–‘–Š‡•›•–‡
# Step Expected Result
1 User clicks Change Password button ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
2 User enters their current password cor- —””‡–’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
”‡…–Ž›
3 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž› ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
4 User re-enters their new password correct- ‡™”‡Ǧ‡–‡”‡†’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›-
Ž› bols
5 User clicks OK ›•–‡†‹•’Žƒ›•‡••ƒ‰‡Dzƒ••™‘”†…Šƒ‰‡†•—……‡••ˆ—ŽŽ›dz

Table B.49 — Test cases for use case testing continued

Use Case Name …Šƒ‰‡̴’ƒ••™‘”†

Test Case Name Alternative Flow – Existing Password Incorrect
Description •‡”ƒ––‡’–•–‘…Šƒ‰‡’ƒ••™‘”†„—–‡–‡”•–Š‡‹”…—””‡–’ƒ••™‘”†‹…‘””‡…–Ž›
Actors User
Test Coverage Item TCOND2
Use Case Steps Covered 1, 2, 3.1, 3.2, 3.3, 3.4, 3.5
Preconditions •‡”‹•ƒŽ”‡ƒ†›Ž‘‰‰‡†‹–‘–Š‡•›•–‡
# Step Expected Result
1 User clicks Change Password button ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
2 •‡”‡–‡”•…—””‡–’ƒ••™‘”†‹…‘””‡…–Ž› —””‡–’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
3 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž› ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•

© ISO/IEC 2015 – All rights reserved 97

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table B.49 (continued)

4 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž› ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
5 User clicks OK ›•–‡†‹•’Žƒ›•‡””‘”‡••ƒ‰‡Dz—””‡–’ƒ••™‘”†‡–‡”‡†
‹…‘””‡…–Ž›ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz

Table B.50 — Test cases for use case testing continued

Use Case Name …Šƒ‰‡̴’ƒ••™‘”†

Test Case Name Alternative Flow – New Password Less Than 8 Characters
Description User attempts to change password but enters less than 8 characters for password
Actors User
Test Coverage Item TCOND3
Use Case Steps Covered 1, 2, 3, 4.1, 4.2, 4.3
Preconditions •‡”‹•ƒŽ”‡ƒ†›Ž‘‰‰‡†‹–‘–Š‡•›•–‡
# Step Expected Result
1 User clicks Change Password button ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
2 User enters their current password cor- —””‡–’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
”‡…–Ž›
3 User enters a new password that is less ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
than 8 characters long
4 User clicks OK ›•–‡†‹•’Žƒ›•ƒ‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†—•–„‡ƒ–
Ž‡ƒ•–ͺ…Šƒ”ƒ…–‡”•Ž‘‰ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz

Table B.51 — Test cases for use case testing continued

Use Case Name …Šƒ‰‡̴’ƒ••™‘”†

Test Case Name Alternative Flow – New Password Same as Current Password
Description User attempts to change password but enters new password matching old pass-
word
Actors User
Test Coverage Item TCOND4
Use Case Steps Covered 1, 2, 3, 5.1, 5.2, 5.3
Preconditions •‡”‹•ƒŽ”‡ƒ†›Ž‘‰‰‡†‹–‘–Š‡•›•–‡
# Step Expected Result
1 User clicks Change Password button ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
2 User enters their current password cor- —””‡–’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
”‡…–Ž›
3 User enters a new password that is the ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
same as their current password
4 User clicks OK ›•–‡†‹•’Žƒ›•‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†—•–‘–„‡
–Š‡•ƒ‡ƒ•…—””‡–’ƒ••™‘”†ǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz

Table B.52 — Test cases for use case testing continued

Use Case Name …Šƒ‰‡̴’ƒ••™‘”†

Test Case Name Ž–‡”ƒ–‹˜‡ Ž‘™Ȃ‡™ƒ••™‘”†•‘‘–ƒ–…Š
Description User attempts to change password but their new passwords do not match
Actors User

98 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.52 (continued)

Test Coverage Item TCOND5
Use Case Steps Covered 1, 2, 3, 4, 6.1, 6.2, 6.3
Preconditions •‡”‹•ƒŽ”‡ƒ†›Ž‘‰‰‡†‹–‘–Š‡•›•–‡
# Step Expected Result
1 User clicks Change Password button ›•–‡†‹•’Žƒ›•Šƒ‰‡ƒ••™‘”†•…”‡‡
2 User enters their current password cor- —””‡–’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
”‡…–Ž›
3 •‡”‡–‡”•–Š‡‹”‡™’ƒ••™‘”†…‘””‡…–Ž› ‡™’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
4 User re-enters new password that does not ‡Ǧ‡–‡”‡†’ƒ••™‘”†‹•ƒ•‡†™‹–Šƒ•–‡”‹•ȋȗȌ•›„‘Ž•
match new password entered at step 3
5 User clicks OK ›•–‡†‹•’Žƒ›•‡””‘”‡••ƒ‰‡Dz‡™’ƒ••™‘”†•†‘‘–
ƒ–…ŠǤŽ‡ƒ•‡–”›ƒ‰ƒ‹Ǥdz

B.2.9.3.7 Step 5: Assemble Test Sets (TD5)

‡•–•…‘—Ž†„‡‰”‘—’‡†ƒ……‘”†‹‰–‘™Š‡–Š‡”–Š‡›…‘˜‡”–Š‡–›’‹…ƒŽ‘”ƒŽ–‡”ƒ–‹˜‡•…‡ƒ”‹‘•ǣ

TS1: TEST CASE 1.

TS2: TEST CASES 2, 3, 4, 5.

B.2.9.3.8 Step 6: Derive Test Procedures (TD6)

Ž›‘‡’”‘…‡†—”‡‹•”‡“—‹”‡†ƒ•ˆ‘ŽŽ‘™•ǣ

ͳǣ…‘˜‡”‹‰–Š‡–‡•–…ƒ•‡‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

ʹǣ…‘˜‡”‹‰–Š‡–‡•–…ƒ•‡•‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.9.3.9 Use Case Testing Coverage

Using the formula for calculating scenario test coverage provided in 6.2.9 and the test coverage items
derived above:
5
Coverage(u sec ase ) = × 100% = 100%
5

B.2.10 Random Testing

B.2.10.1 Introduction

The aim of random testing is to derive a set of test cases that cover the input parameters of a test item
using values that are selected according to a chosen input distribution. This technique requires no
’ƒ”–‹–‹‘‹‰‘ˆ–Š‡‹’—–†‘ƒ‹‘ˆ–Š‡–‡•–‹–‡ǡ„—–•‹’Ž›”‡“—‹”‡•‹’—–˜ƒŽ—‡•–‘„‡…Š‘•‡ˆ”‘–Š‹•
input domain at random.

ǤʹǤͳͲǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider a test item that transforms coordinates, with the following test basis:
The component shall transform the Cartesian coordinates (x,y) for screen position into their polar equivalent
(r,H) using the equations: r= sqrt (x2+y2) and cos H = x/r. The origin of the Cartesian coordinates and the
pole of the polar coordinates shall be the centre of the screen and the x-axis shall be considered the initial

© ISO/IEC 2015 – All rights reserved 99

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

line for the polar coordinates progressing counter-clockwise. All inputs and outputs shall be represented as
ϔ‹š‡†Ǧ’‘‹–—„‡”•™‹–Š„‘–Šƒ”ƒ‰‡ƒ†ƒ’”‡…‹•‹‘ǤŠ‡•‡•ŠƒŽŽ„‡ǣ
Inputs
x - range -320..+320, in increments of 1/26
y - range -240..+240, in increments of 1/27
Outputs
r - range 0..400, in increments of 1/26
H - range 0..((2*pi)-1/26), in increments of 1/26

B.2.10.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: transform coordinates function

B.2.10.4 Step 2: Derive Test Conditions (TD2)

The test conditions in random testing are the domain of all possible inputs from which test input values
…ƒ„‡•‡Ž‡…–‡†ˆ‘”‡ƒ…Š‹’—–’ƒ”ƒ‡–‡”ǤŠ‡›ƒ”‡ǣ

TCOND1: x - range -320..+320, in increments of 1/26 (for FS1)

TCOND2: ›Ǧ”ƒ‰‡ǦʹͶͲǤǤΪʹͶͲǡ‹‹…”‡‡–•‘ˆͳȀʹ7 (for FS1)

B.2.10.5 Step 3: Derive Test Coverage Items (TD3)

There are no recognised test coverage items from random testing.

B.2.10.6 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•…ƒ‘™„‡…‘•–”—…–‡†„›ϐ‹”•–…Š‘‘•‹‰ƒ‹’—–†‹•–”‹„—–‹‘ƒ†–Š‡ƒ’’Ž›‹‰–Šƒ–‹’—–
distribution to each test condition and determining the expected result of each test case (shown as
Ǯ‘—–’—–ǯˆ‘”–Š‡–™‘‘—–’—–’ƒ”ƒ‡–‡”•Ǯ”ǯƒ†Ǯǯ‹–Š‡–ƒ„Ž‡„‡Ž‘™ȌǤ‹…‡‘‹ˆ‘”ƒ–‹‘‹•ƒ˜ƒ‹Žƒ„Ž‡
about the operational distribution of the input parameters to the test item in this example, a uniform
†‹•–”‹„—–‹‘‹•…Š‘•‡Ǥ ”‘–Š‡†‡ϐ‹‹–‹‘•™‡…ƒ•‡‡–Šƒ–‹ƒ›‘‡”ƒ†‘Ž›…Š‘•‡‹’—–ˆ‘”š…ƒ
take one of 41,024 values (641 x 26Ȍǡ™Š‹Ž‡›…ƒ–ƒ‡‘‡‘ˆ͸ͳǡͷ͸ͺ˜ƒŽ—‡•ȋͶͺͳšʹ7). Care should be taken
if using an expected operational distribution rather than a uniform distribution. An expected distribution
that ignores parts of the input domain can lead to unexpected error conditions being left untested.
Since each test case must include selection of a random test input value from the test conditions for
„‘–Ššƒ†›ǡ‡ƒ…Š–‡•–…ƒ•‡™‹ŽŽ…‘˜‡”„‘–Š–‡•–…‘†‹–‹‘•Ǥƒ—‹ˆ‘”†‹•–”‹„—–‹‘ǡƒŽŽ‹’—–˜ƒŽ—‡•
™‹–Š‹–Š‡†‡ϐ‹‡”ƒ‰‡•‘ˆšƒ†›Šƒ˜‡‡“—ƒŽ’”‘„ƒ„‹Ž‹–›‘ˆ„‡‹‰•‡Ž‡…–‡†ƒ•‹’—–•‹–‘–Š‡–‡•–…ƒ•‡Ǥ
‘”‡šƒ’Ž‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰ˆ‘—”–‡•–…ƒ•‡•…‘—Ž†„‡†‡ϐ‹‡†Ǥ

Table B.53 — Test cases for random testing

Test Case 1 2 3 4
Input (x) -126.125 11.015625 283.046875 -99.109375
’—–ȋ›Ȍ 238.046875 78.03125 -156.054688 -9.0625
Test Condition TCOND1 TCOND1 TCOND1 TCOND1
TCOND2 TCOND2 TCOND2 TCOND2
—–’—–”…ƒŽ…—Žƒ–‡†ƒ•ȋ”α•“”–ȋš2Ϊ›2)) 269.395305 78.804949 323.216025 99.522847

100 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table B.53 (continued)

Test Case 1 2 3 4
—–’—–…ƒŽ…—Žƒ–‡†ƒ•ȋ…‘•αšȀ”ȌȌ 2.058024 1.430554 0.503870 3.050407

B.2.10.7 Step 5: Assemble Test Sets (TD5)

ˆ™‡ƒ••—‡–Šƒ–ƒŽŽ–‡•–…ƒ•‡•—•–„‡‡š‡…—–‡†ƒ—ƒŽŽ›ƒ†‹–Š‡‘”†‡”–Šƒ––Š‡›™‡”‡†‡ϐ‹‡†‹
–Š‡–‡•–…ƒ•‡–ƒ„Ž‡ǡ–Š‡™‡…‘—Ž††‡ϐ‹‡‘‡–‡•–•‡–ƒ•ˆ‘ŽŽ‘™•ǣ
ͳǣƒ—ƒŽ‡•–‹‰Ȃͳǡʹǡ͵ǡͶǤ

B.2.10.8 Step 6: Derive Test Procedures (TD6)

‹…‡–Š‡”‡‹•‘Ž›‘‡–‡•–•‡––‘‡š‡…—–‡ǡ‹–‹••‡•‹„Ž‡–‘†‡ϐ‹‡‘‡–‡•–’”‘…‡†—”‡ƒ•ˆ‘ŽŽ‘™•ǣ
ͳǣƒ—ƒŽ–‡•–‹‰ǡ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

B.2.10.9 Random Testing Coverage

As stated in 6.2.10ǡ –Š‡”‡ ‹• …—””‡–Ž› ‘ ‹†—•–”› ƒ‰”‡‡† ƒ’’”‘ƒ…Š ˆ‘” …ƒŽ…—Žƒ–‹‰ …‘˜‡”ƒ‰‡ ‘ˆ –‡•–
coverage items for random testing.

B.2.10.10 Automating Random Testing

ƒ†‘ –‡•–‹‰ ƒ› „‡ ’‡”ˆ‘”‡† ‡‹–Š‡” ƒ—ƒŽŽ› ‘” —•‹‰ ƒ—–‘ƒ–‹‘Ǥ ƒ†‘ –‡•–‹‰ ‹• ‘•–
…‘•–Ǧ‡ˆˆ‡…–‹˜‡™Š‡ˆ—ŽŽ›ƒ—–‘ƒ–‡†ƒ•–Š‡˜‡”›ƒ›–‡•–•…ƒ„‡”—™‹–Š‘—–ƒ—ƒŽ‹–‡”˜‡–‹‘Ǥ
However, to achieve full automation it must be possible to:
Ȅ ƒ—–‘ƒ–‹…ƒŽŽ›‰‡‡”ƒ–‡”ƒ†‘–‡•–‹’—–•Ǣƒ†
Ȅ ‡‹–Š‡”ƒ—–‘ƒ–‹…ƒŽŽ›‰‡‡”ƒ–‡‡š’‡…–‡†”‡•—Ž–•ˆ”‘–Š‡–‡•–„ƒ•‹•Ǣ‘”
Ȅ ƒ—–‘ƒ–‹…ƒŽŽ›…Š‡…–‡•–‘—–’—–•ƒ‰ƒ‹•––Š‡–‡•–„ƒ•‹•Ǥ
Š‡ƒ—–‘ƒ–‹…‰‡‡”ƒ–‹‘‘ˆ”ƒ†‘–‡•–‹’—–˜ƒŽ—‡•‹•‘–†‹ˆϐ‹…—Ž–—•‹‰ƒ’•‡—†‘Ǧ”ƒ†‘—„‡”
‰‡‡”ƒ–‘”ƒ•Ž‘‰ƒ•–Š‡–‡•–‹–‡ǯ•‹’—–•ƒ”‡™‡ŽŽǦ†‡ϐ‹‡†Ǥˆ–Š‡–‡•–‹’—–˜ƒŽ—‡•ƒ”‡’”‘†—…‡†—•‹‰ƒ
’•‡—†‘Ǧ”ƒ†‘—„‡”‰‡‡”ƒ–‘”ǡ–Š‡–Š‡•‡˜ƒŽ—‡•†‘‘–‡‡†–‘„‡”‡…‘”†‡†‡š’Ž‹…‹–Ž›ƒ•–Š‡•ƒ‡
•‡–…ƒ„‡”‡’”‘†—…‡†ǤŠ‹•‹•‘”ƒŽŽ›’‘••‹„Ž‡‹ˆƒDz•‡‡†dz˜ƒŽ—‡Šƒ•„‡‡—•‡†–‘’”‹‡–Š‡’•‡—†‘Ǧ
random number generator and this value is recorded.
The automatic generation of expected outputs or the automatic checking of outputs, is however more
’”‘„Ž‡ƒ–‹…Ǥ
‡‡”ƒŽŽ›‹–‹•‘–’”ƒ…–‹…ƒ„Ž‡–‘ƒ—–‘ƒ–‹…ƒŽŽ›‰‡‡”ƒ–‡‡š’‡…–‡†‘—–’—–•‘”ƒ—–‘ƒ–‹…ƒŽŽ›
check outputs against the test basis, however for certain test items it is possible, such as where:
Ȅ –”—•–‡† ‹†‡’‡†‡–Ž›Ǧ’”‘†—…‡† •‘ˆ–™ƒ”‡ –Šƒ– ’‡”ˆ‘”• –Š‡ •ƒ‡ ˆ—…–‹‘ ƒ• –Š‡ –‡•– ‹–‡ ‹•
ƒ˜ƒ‹Žƒ„Ž‡ȋ’”‡•—ƒ„Ž›‘–‡‡–‹‰–Š‡•ƒ‡…‘•–”ƒ‹–••—…Šƒ••’‡‡†‘ˆ’”‘…‡••‹‰ǡ‹’Ž‡‡–ƒ–‹‘
language, etc.);
Ȅ –Š‡–‡•–‹•…‘…‡”‡†•‘Ž‡Ž›™‹–Š™Š‡–Š‡”–Š‡–‡•–‹–‡…”ƒ•Š‡•‘”‘–ȋ•‘–Š‡‡š’‡…–‡†”‡•—Ž–‹•
“not to crash”);
Ȅ –Š‡ ƒ–—”‡ ‘ˆ –Š‡ –‡•– ‹–‡ǯ• ‘—–’—– ƒ‡• ‹– ”‡Žƒ–‹˜‡Ž› ‡ƒ•› –‘ …Š‡… –Š‡ ”‡•—Ž–Ǥ  ‡šƒ’Ž‡ ‘ˆ
–Š‹•‹•ƒ•‘”–ˆ—…–‹‘™Š‡”‡‹–‹•ƒ•‹’Ž‡–ƒ•–‘ƒ—–‘ƒ–‹…ƒŽŽ›…Š‡…–Šƒ––Š‡‘—–’—–•Šƒ˜‡„‡‡
•‘”–‡†…‘””‡…–Ž›Ǣ
Ȅ ‹– ‹• ‡ƒ•› –‘ ‰‡‡”ƒ–‡ ‹’—–• ˆ”‘ –Š‡ ‘—–’—–• ȋ—•‹‰ –Š‡ ‹˜‡”•‡ ‘ˆ –Š‡ –‡•– ‹–‡ǯ• ˆ—…–‹‘ȌǤ 
‡šƒ’Ž‡‘ˆ–Š‹•‹•ƒ•“—ƒ”‡”‘‘–ˆ—…–‹‘™Š‡”‡•‹’Ž›•“—ƒ”‹‰–Š‡‘—–’—–•Š‘—Ž†’”‘†—…‡–Š‡‹’—–Ǥ
In the example in B.2.10.2ǡ–Š‡…‘‘”†‹ƒ–‡–”ƒ•ˆ‘”ƒ–‹‘–‡•–‹–‡…ƒ„‡…Š‡…‡†ƒ—–‘ƒ–‹…ƒŽŽ›—•‹‰
–Š‡ ‹˜‡”•‡ ˆ—…–‹‘ ƒ’’”‘ƒ…ŠǤ  –Š‹• …ƒ•‡ǡ ”…‘•αš …ƒ „‡ ‘„–ƒ‹‡† †‹”‡…–Ž› ˆ”‘ –Š‡ –‡•– „ƒ•‹• ˆ‘”
© ISO/IEC 2015 – All rights reserved 101
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

–Š‡–‡•–‹–‡Ǥ›•‘‡ƒƒŽ›•‹•”•‹α›…ƒƒŽ•‘„‡†‡†—…‡†Ǥˆ–Š‡•‡–™‘‡“—ƒ–‹‘•ƒ”‡•ƒ–‹•ϐ‹‡†–‘ƒ

”‡ƒ•‘ƒ„Ž‡—‡”‹…ƒŽ–‘Ž‡”ƒ…‡–Š‡–Š‡–‡•–‹–‡Šƒ•–”ƒ•ˆ‘”‡†–Š‡…‘‘”†‹ƒ–‡•…‘””‡…–Ž›Ǥ
Even when full automation of random testing is not practicable its use should still be considered as it
†‘‡•‘–…ƒ””›–Š‡Žƒ”‰‡‘˜‡”Š‡ƒ†‘ˆ†‡•‹‰‹‰–‡•–…ƒ•‡•ƒ•”‡“—‹”‡†„›–Š‡‘Ǧ”ƒ†‘–‡…Š‹“—‡•Ǥ
‘” –‡•– ‹–‡• ™‹–Š Žƒ”‰‡” ‹’—– •‡–• –Šƒ –Š‹• •ƒŽŽ ‡šƒ’Ž‡ –Š‡ Dz›„‘Ž‹… ’—– ––”‹„—–‡
Decomposition” (SIAD) tree (Cho 1987) is a useful method for organising the input domain for random
sampling before test case design.

102 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Annex C
(informative)

Guidelines and Examples for the Application of Structure-Based

Test Design Techniques

C.1 Guidelines and Examples for Structure-Based Testing

C.1.1 Overview
This annex provides guidance and examples on the structure-based test design techniques described
in 5.3 and 6.3Ǥ ƒ…Š ‡šƒ’Ž‡ ˆ‘ŽŽ‘™• –Š‡ ‡•– ‡•‹‰ ƒ† ’Ž‡‡–ƒ–‹‘ ”‘…‡•• –Šƒ– ‹• †‡ϐ‹‡† ‹
ȀȀ ʹͻͳͳͻǦʹǤ  ˜ƒ”‹‡–› ‘ˆ ƒ’’Ž‹…ƒ–‹‘• ƒ† ’”‘‰”ƒ‹‰ Žƒ‰—ƒ‰‡• ƒ”‡ —•‡† ‹ –Š‡•‡
examples. Although each example is applied in a structure-based testing context, as stated in 5.1, in
’”ƒ…–‹…‡‘•–‘ˆ–Š‡–‡…Š‹“—‡•†‡ϐ‹‡†‹–Š‹•’ƒ”–‘ˆȀȀʹͻͳͳͻ…ƒ„‡—•‡†‹–‡”…Šƒ‰‡ƒ„Ž›Ǥ

C.2 Structure-Based Test Design Technique Examples

C.2.1 Statement Testing

C.2.1.1 Introduction

The aim of statement testing is to derive a set of test cases that cover the statements of the test item
according to a chosen level of statement coverage. This structural test design technique is based upon
the decomposition of the test item into constituent statements.
The two principal questions to consider are:
— what is a statement?
— which statements are executable?
‰‡‡”ƒŽƒ•–ƒ–‡‡–•Š‘—Ž†„‡ƒƒ–‘‹…ƒ…–‹‘Ǣ–Šƒ–‹•ƒ•–ƒ–‡‡–•Š‘—Ž†„‡‡š‡…—–‡†…‘’Ž‡–‡Ž›‘”
not at all. For instance:
IF a THEN b ENDIF

is considered as more than one statement since b ƒ› ‘” ƒ› ‘– „‡ ‡š‡…—–‡† †‡’‡†‹‰ —’‘ –Š‡
condition DǤ Š‡ †‡ϐ‹‹–‹‘ ‘ˆ statement used for statement testing need not be the one used in the
Žƒ‰—ƒ‰‡†‡ϐ‹‹–‹‘Ǥ
We would expect statements which are associated with machine code to be regarded as executable. For
instance, we would expect all of the following to be regarded as executable:
— assignments;
— loops and selections;
— procedure and function calls;
— variable declarations with explicit initializations;
Ȅ †›ƒ‹…ƒŽŽ‘…ƒ–‹‘‘ˆ˜ƒ”‹ƒ„Ž‡•–‘”ƒ‰‡‘ƒŠ‡ƒ’Ǥ

© ISO/IEC 2015 – All rights reserved 103

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

However, most other variable declarations can be regarded as non-executable. Consider the
following code:
D
LIE^
F
`
G

›–‡•–…ƒ•‡™‹–Š„™‹ŽŽƒ…Š‹‡˜‡ˆ—ŽŽ•–ƒ–‡‡–…‘˜‡”ƒ‰‡Ǥ‘–‡–Šƒ–ˆ—ŽŽ•–ƒ–‡‡–…‘˜‡”ƒ‰‡…ƒ
be achieved without exercising with b FALSE.

ǤʹǤͳǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the following test item in the Ada programming language, which is designed to categorise
positive integers into prime and non-prime, and to give factors for those which are non-prime:
5($'1XP
:+,/(127(QGRI)LOH'2
3ULPH 758(
)25)DFWRU 721XP',9'2
,)1XP1XP',9)DFWRU )DFWRU 7+(1
:5,7()DFWRUCLVDIDFWRURI¶1XP
3ULPH )$/6(
(1',)
(1')25
,)3ULPH 758(7+(1
:5,7(1XPCLVSULPH¶
(1',)
5($'1XP
(1':+,/(
:5,7(C(QGRISULPHQXPEHUSURJUDP¶
C.2.1.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
ͳǣ†‡–‹ˆ›’”‹‡ƒ†‘Ǧ’”‹‡—„‡”•ˆ—…–‹‘

C.2.1.4 Step 2: Derive Test Conditions (TD2)

‡•– …‘†‹–‹‘• ‹ •–ƒ–‡‡– –‡•–‹‰ ƒ”‡ –Š‡ ‡š‡…—–ƒ„Ž‡ •–ƒ–‡‡–• ‹ –Š‡ …‘†‡Ǥ › —„‡”‹‰ ‡ƒ…Š
Ž‹‡‘ˆ…‘†‡ǡ–Š‹•‹ŽŽ—•–”ƒ–‡•–‡•–…‘†‹–‹‘—„‡”•Ǥ ‘”‡šƒ’Ž‡ǡ•–ƒ–‡‡–ͳƒŽŽ‘™•†‡ϐ‹‹–‹‘‘ˆ‘‡
test condition:

TCOND1: 5($'1XP Statement 1 (for FS1)

Š‡ ”‡ƒ‹‹‰ –‡•– …‘†‹–‹‘• …ƒ „‡ †‡ϐ‹‡† ™‹–Š‘—– ”‡’‡ƒ–‹‰ –Š‡ •‘—”…‡ …‘†‡ ‡ƒ…Š –‡•– …‘†‹–‹‘
relates to:

TCOND2: Statement 2 (for FS1)

TCOND3: Statement 3 (for FS1)

TCOND4: Statement 4 (for FS1)

TCOND5: Statement 5 (for FS1)

TCOND6: Statement 6 (for FS1)

TCOND7: Statement 7 (for FS1)

TCOND8: Statement 10 (for FS1)

104 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

TCOND9: Statement 11 (for FS1)

TCOND10: Statement 13 (for FS1)

TCOND11: Statement 15 (for FS1)

C.2.1.5 Step 3: Derive Test Coverage Items (TD3)

The test coverage items in statement testing are the same as the test conditions:

TCOVER1: Statement 1 (for TCOND1)

TCOVER2: Statement 2 (for TCOND2)

TCOVER3 Statement 3 (for TCOND3)

TCOVER4: Statement 4 (for TCOND4)

TCOVER5: Statement 5 (for TCOND5)

TCOVER6: Statement 6 (for TCOND6)

TCOVER7: Statement 7 (for TCOND7)

TCOVER8: Statement 10 (for TCOND8)

TCOVER9: Statement 11 (for TCOND9)

TCOVER10: Statement 13 (for TCOND13)

TCOVER11: Statement 15 (for TCOND11)

C.2.1.6 Step 4: Derive Test Cases (TD4)

•–ƒ–‡‡––‡•–‹‰ǡ‡ƒ…Š•–ƒ–‡‡–—•–„‡…‘˜‡”‡†„›ƒ–Ž‡ƒ•–‘‡–‡•–…ƒ•‡Ǥ‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†
„›ϐ‹”•–‹†‡–‹ˆ›‹‰•—„Ǧ’ƒ–Š•‹–Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’Š–Šƒ–‡š‡…—–‡‘‡‘”‘”‡‡š‡…—–ƒ„Ž‡•–ƒ–‡‡–•
–Šƒ–Šƒ˜‡‘–›‡–„‡‡…‘˜‡”‡†„›ƒ–‡•–…ƒ•‡ǤŠ‡‹’—–•–‘‡š‡…—–‡–Š‡•—„Ǧ’ƒ–Šƒ”‡–Š‡‹†‡–‹ϐ‹‡†ǡ
along with the expected result. This process is repeated until the required level of test coverage is
ƒ…Š‹‡˜‡†Ǥ–Š‹•’ƒ”–‹…—Žƒ”‡šƒ’Ž‡ǡ‘Ž›‘‡–‡•–…ƒ•‡‹•”‡“—‹”‡†–‘…‘˜‡”ƒŽŽ•–ƒ–‡‡–•‹–Š‡…‘†‡ǡ
„‡…ƒ—•‡–Š‡‹–‡”ƒ–‹‘‹–Š‡…‘†‡–Šƒ–ƒŽŽ‘™•‹––‘„‡…‘˜‡”‡†„›Œ—•––™‘‹’—–˜ƒŽ—‡•ȋ‹Ǥ‡Ǥ–‘ƒ…Š‹‡˜‡
100% statement coverage).

Table C.1 — Test cases for statement testing

Test Case Input Expected Result Test Coverage Items

1 2 2 is prime 1, 2, 3, 4, 9, 10, 11, 12, 13, 14
4 2 is a factor of 4 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14
EOF End of prime number program 2, 15

NOTE In this example, there is one test case, but three separate sets of input values and expected results, as
there is an iteration included in the code.

C.2.1.7 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡‹•‘Ž›‘‡–‡•–…ƒ•‡ǡ‹–…ƒ„‡’Žƒ…‡†‹–Š‡‘‡–‡•–•‡–ƒ•ˆ‘ŽŽ‘™•ǣ
TS1: TEST CASE 1.

© ISO/IEC 2015 – All rights reserved 105

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

C.2.1.8 Step 6: Derive Test Procedures (TD6)

Ž›‘‡’”‘…‡†—”‡‹•”‡“—‹”‡†ƒ•ˆ‘ŽŽ‘™•ǣ
TP1: covering the test case in TS1.

C.2.1.9 Statement Testing Coverage

Using the formula provided in 6.3.1 and the test coverage items derived above:
11
Coverage( statement ) = × 100% = 100%
11
Thus, 100% coverage of test coverage items for statement testing has been achieved.

C.2.2 Branch / Decision Testing

C.2.2.1 Introduction

”ƒ…Š ƒ† ‡…‹•‹‘ ‘˜‡”ƒ‰‡ ƒ”‡ …Ž‘•‡Ž› ”‡Žƒ–‡†Ǥ ‘” –‡•– ‹–‡• ™‹–Š ‘‡ ‡–”› ’‘‹– ͳͲͲΨ ”ƒ…Š
‘˜‡”ƒ‰‡‹•‡“—‹˜ƒŽ‡––‘ͳͲͲΨ‡…‹•‹‘‘˜‡”ƒ‰‡ǡƒŽ–Š‘—‰ŠŽ‘™‡”Ž‡˜‡Ž•‘ˆ…‘˜‡”ƒ‰‡ƒ›‘–„‡–Š‡
same. Both levels of coverage will be illustrated with one example.

ǤʹǤʹǤʹ ’‡…‹ϐ‹…ƒ–‹‘

The component shall determine the position of a word in a table of words ordered alphabetically. Apart
from the word and table, the component shall also be passed the number of words in the table to be
searched. The component shall return the position of the word in the table (starting at zero) if it is
ˆ‘—†ǡ‘–Š‡”™‹•‡‹–•ŠƒŽŽ”‡–—”DzǦͷdzǤ
The corresponding code is drawn from (Kernighan and Richie 1998). The three decisions are highlighted:
LQWELQVHDUFKFKDU ZRUGVWUXFWNH\WDE>@LQWQ^
LQWFRQG
LQWORZKLJKPLG
ORZ 
KLJK Q
while (low <= high)^
PLG ORZKLJK
LIFRQG VWUFPSZRUGWDE>PLG@ZRUG
KLJK PLG
HOVHif (cond > 0)
ORZ PLG
HOVH
UHWXUQPLG
`
UHWXUQ
`
C.2.2.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: binsearch function

C.2.2.4 Step 2: Derive Test Conditions (TD2)

C.2.2.4.1 Options for Derivation of Test Conditions

Š‡ ‹†‡–‹ϐ‹…ƒ–‹‘ ‘ˆ –‡•– …‘†‹–‹‘• ˆ‘” „”ƒ…ŠȀ†‡…‹•‹‘ –‡•–‹‰ ƒ› „‡ †‡‘•–”ƒ–‡† „› …”‡ƒ–‹‰ ƒ
…‘–”‘ŽϐŽ‘™‰”ƒ’Š‘ˆ–Š‡’”‘‰”ƒǤŠ‡ϐ‹”•–•–‡’–‘…‘•–”—…–‹‰ƒ…‘–”‘ŽϐŽ‘™‰”ƒ’Šˆ‘”ƒ’”‘…‡†—”‡‹•
to divide it into basic blocks. These are sequences of instructions with no branches into the block (except

106 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

to the beginning) and no branches out of the block (except at the end). The statements within each basic
block will be executed together or not at all. The program above has the following basic blocks:
LQWELQVHDUFKFKDU ZRUGVWUXFWNH\WDE>@LQWQ^
LQWFRQG
LQWORZKLJKPLG
%ORZ 
KLJK Q
%ZKLOHORZ KLJK^
%PLG ORZKLJK
LIFRQG VWUFPSZRUGWDE>PLG@ZRUG
% KLJK PLG
% HOVHLIFRQG!
% ORZ PLG
%HOVH
UHWXUQPLG
% `
%UHWXUQ
`
…‘–”‘ŽϐŽ‘™‰”ƒ’Šƒ›„‡…‘•–”—…–‡†„›ƒ‹‰‡ƒ…Š„ƒ•‹…„Ž‘…ƒ‘†‡ƒ††”ƒ™‹‰ƒƒ”…ˆ‘”‡ƒ…Š
possible transfer of control from one basic block to another. These are the possible transfers of control:

ͳ՜ȝʹ ͵՜Ͷ ͷ՜͸ ͸՜ͺ

ʹ՜͵ ͵՜ͷ ͷ՜͹ ͺ՜ʹ
ʹ՜ͻ Ͷ՜ͺ
This results in the graph presented in ϐ‹‰—”‡ ǤͳǤ Š‡ ‰”ƒ’Š Šƒ• ‘‡ ‡–”› ’‘‹–ǡ ͳǡ ƒ† –™‘ ‡š‹–
points, B7 and B9.

B1 B2 B3 B4 B8

B9 B5 B6

‹‰—”‡ǤͳȄ‘–”‘ŽϐŽ‘™‰”ƒ’Šˆ‘”„‹•‡ƒ”…Š

ˆ…‘—”•‡ǡ–Š‡ƒ„‘˜‡…‘–”‘ŽϐŽ‘™‰”ƒ’Š™‘—Ž†‘–‡…‡••ƒ”‹Ž›„‡…‘•–”—…–‡†„›Šƒ†ǡ„—–ƒ–‘‘Ž™‘—Ž†
‘”ƒŽŽ›„‡—•‡†–‘•Š‘™™Š‹…Š†‡…‹•‹‘•Ȁ„”ƒ…Š‡•Šƒ˜‡„‡‡‡š‡…—–‡†Ǥ
The test conditions for branch coverage will be different from those for decision coverage. This is
demonstrated under steps 2a and 2b below.

C.2.2.4.2 Option 2a: Derive Test Conditions for Branch Coverage (TD2)

For branch coverage, the test conditions (BRANCH-TCOND) are the branches (arcs) that are represented
„›ƒ””‘™•‹–Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’ŠǤŠ‡”‡ƒ”‡–‡‹–‘–ƒŽǡƒ•ˆ‘ŽŽ‘™•ǣ

BRANCH-TCOND1: ͳ՜ʹ (for FS1)

BRANCH-TCOND2: ʹ՜͵ (for FS1)

BRANCH-TCOND3: ʹ՜ͻ (for FS1)

© ISO/IEC 2015 – All rights reserved 107

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

BRANCH-TCOND4: ͵՜Ͷ (for FS1)

BRANCH-TCOND5: ͵՜ͷ (for FS1)

BRANCH-TCOND6: Ͷ՜ͺ (for FS1)

BRANCH-TCOND7: ͷ՜͸ (for FS1)

BRANCH-TCOND8: ͷ՜͹ (for FS1)

BRANCH-TCOND9: ͸՜ͺ (for FS1)

BRANCH-TCOND10: ͺ՜ʹ (for FS1)

C.2.2.4.3 Option 2b: Derive Test Conditions for Decision Coverage (TD2)

For decision coverage, the test conditions (DECISION-TCOND) are the decisions represented as nodes in
–Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’Š–Šƒ–Šƒ˜‡‘”‡–Šƒ‘‡‡š‹–ƒ””‘™Ǥ–Š‹•‡šƒ’Ž‡ǡ–Š‡”‡ƒ”‡–Š”‡‡–‡•–…‘†‹–‹‘•ǣ

DECISION-TCOND1: B2 (for FS1)

DECISION-TCOND2: B3 (for FS1)

DECISION-TCOND3: B5 (for FS1)

C.2.2.5 Step 3: Derive Test Coverage Items (TD3)

C.2.2.5.1 Options for Derivation of Test Coverage Items

The test coverage items for branch coverage will be different from those for decision coverage. This is
demonstrated under steps 3a and 3b below.

C.2.2.5.2 Option 3a: Derive Test Coverage Items for Branch Coverage (TD3)

‘” „”ƒ…Š …‘˜‡”ƒ‰‡ǡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ –Š‡ „”ƒ…Š‡• ‹ –Š‡ …‘–”‘Ž ϐŽ‘™ ‰”ƒ’Šǡ ™Š‹…Š
are the same as the test conditions. In this example there are ten test coverage items for branch
coverage, as follows:

BRANCH-TCOVER1: ͳ՜ʹ (for BRANCH-TCOND1)

BRANCH-TCOVER2: ʹ՜͵ (for BRANCH-TCOND2)

BRANCH-TCOVER3: ʹ՜ͻ (for BRANCH-TCOND3)

BRANCH-TCOVER4: ͵՜Ͷ (for BRANCH-TCOND4)

BRANCH-TCOVER5: ͵՜ͷ (for BRANCH-TCOND5)

BRANCH-TCOVER6: Ͷ՜ͺ (for BRANCH-TCOND6)

BRANCH-TCOVER7: ͷ՜͸ (for BRANCH-TCOND7)

BRANCH-TCOVER8: ͷ՜͹ (for BRANCH-TCOND8)

BRANCH-TCOVER9: ͸՜ͺ (for BRANCH-TCOND9)

BRANCH-TCOVER10: ͺ՜ʹ (for BRANCH-TCOND10)

108 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

C.2.2.5.3 Option 3b: Derive Test Coverage Items for Decision Coverage (TD3)

For decision coverage, the outcomes (i.e. true, false) of each decision are the test coverage items. In this
example, each decision has two outcomes corresponding to the true and false values of the decisions;
therefore there are six test coverage items, as follows:

DECISION-TCOVER1: ʹα–”—‡ (for DECISION-TCOND1)

DECISION-TCOVER2: ʹαˆƒŽ•‡ (for DECISION-TCOND1)

DECISION-TCOVER3: ͵α–”—‡ (for DECISION-TCOND2)

DECISION-TCOVER4: ͵αˆƒŽ•‡ (for DECISION-TCOND2)

DECISION-TCOVER5: ͷα–”—‡ (for DECISION-TCOND3)

DECISION-TCOVER6: ͷαˆƒŽ•‡ (for DECISION-TCOND3)

–‹•‰‡‡”ƒŽŽ›’‘••‹„Ž‡ˆ‘”ƒ†‡…‹•‹‘–‘Šƒ˜‡‘”‡–Šƒ–™‘‘—–…‘‡•ǡ™Š‹…Š™‘—Ž†‹…”‡ƒ•‡–Š‡—„‡”
of test coverage items that need to be derived.

C.2.2.6 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘”„”ƒ…Š–‡•–‹‰ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡
„”ƒ…Š‡•ȋ–‡•–…‘˜‡”ƒ‰‡‹–‡•Ȍ–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰ǡ†‡–‡”‹‹‰‹’—–•–Šƒ–
exercise those sub-paths, determining the expected result of each test, and repeating until the required
Ž‡˜‡Ž ‘ˆ –‡•– …‘˜‡”ƒ‰‡ ‹• ƒ…Š‹‡˜‡†Ǥ ‹‹Žƒ”Ž›ǡ –‡•– …ƒ•‡• ˆ‘” †‡…‹•‹‘ –‡•–‹‰ ƒ”‡ †‡”‹˜‡† „› ‹†‡–‹ˆ›‹‰
…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡†‡…‹•‹‘–Šƒ–Šƒ•‘–„‡‡‡š‡”…‹•‡†ƒ††‡–‡”‹‹‰–Š‡
‹’—–•ƒ†‡š’‡…–‡†‘—–’—–•ˆ‘”‡ƒ…Š–‡•–Ǥ ‘”„‘–Š„”ƒ…Š…‘˜‡”ƒ‰‡ƒ††‡…‹•‹‘…‘˜‡”ƒ‰‡ǡƒ›‹†‹˜‹†—ƒŽ
–‡•–‘ˆ–Š‡–‡•–‹–‡™‹ŽŽ‡š‡”…‹•‡ƒ•—„Ǧ’ƒ–Šƒ†Š‡…‡’‘–‡–‹ƒŽŽ›ƒ›†‡…‹•‹‘•ƒ†„”ƒ…Š‡•Ǥ
‘•‹†‡” ƒ –‡•– …ƒ•‡ ™Š‹…Š ‡š‡…—–‡• –Š‡ •—„Ǧ’ƒ–Š ͳ Ǧε ʹ Ǧε ͻǤ Š‹• …ƒ•‡ ƒ”‹•‡• ™Š‡ αͲǡ –Šƒ– ‹•ǡ
when the table being searched has no entries. This sub-path executes one decision (B2 -> B9) and hence
’”‘˜‹†‡•ͳȀ͸αͳ͸Ǥ͹Ψ…‘˜‡”ƒ‰‡ǤŠ‡’ƒ–Š‡š‡…—–‡•ʹ‘—–‘ˆ–Š‡ͳͲ„”ƒ…Š‡•ǡ‰‹˜‹‰ʹͲΨ…‘˜‡”ƒ‰‡ȋ™Š‹…Š
is not the same as the coverage for decisions).
Consider now a test case which executes the sub-path:
ͳ՜ʹ՜͵՜Ͷ՜ͺ՜ʹ՜͵՜ͷ՜͸՜ͺ՜ʹ՜͵՜ͷ՜͹
Š‹••—„Ǧ’ƒ–Šƒ”‹•‡•™Š‡–Š‡•‡ƒ”…Šϐ‹”•–‘„•‡”˜‡•–Šƒ––Š‡‡–”›‹•‹–Š‡ϐ‹”•–ŠƒŽˆ‘ˆ–Š‡–ƒ„Ž‡ǡ–Š‡
–Š‡•‡…‘†ŠƒŽˆ‘ˆ–Šƒ–ȋ‹Ǥ‡Ǥǡʹ†“—ƒ”–‡”Ȍƒ†–Š‡ϐ‹†•–Š‡‡–”›Ǥ‘–‡–Šƒ––Š‡–™‘–‡•–…ƒ•‡•’”‘˜‹†‡
100% decision and branch coverage.
These test cases are shown below:

Table C.2 — Test cases for binsearch

Inputs
Test Decisions Exercised Test Coverage Items Expected
Case Word Tab n (underlined) Result
1 chas Alf 7 ͳ՜B2 ՜B3 ՜Ͷ՜ͺ՜ BRANCH-TCOVER 2
B2 ՜B3 ՜B5 ՜͸՜ͺ՜ 1, 2, 4, 5, 6, 7, 8, 9, 10 and
Bert
B2 ՜B3 ՜B5 ՜͹
DECISION-TCOVER 1, 3, 4, 5, 6
Chas
Dick
††›
Fred
Geoff

© ISO/IEC 2015 – All rights reserved 109

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table C.2 (continued)

Inputs
Test Decisions Exercised Test Coverage Items Expected
Case Word Tab n (underlined) Result
2 chas Ǯ‡’–› 0 ͳ՜B2 ՜ͻ BRANCH-TCOVER 1, 3 and -1
table’
DECISION-TCOVER 2

”ƒ…Šƒ††‡…‹•‹‘…‘˜‡”ƒ‰‡ƒ”‡„‘–Š‘”ƒŽŽ›‡ƒ•—”‡†—•‹‰ƒ•‘ˆ–™ƒ”‡–‘‘ŽǤ

C.2.2.7 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡ƒ”‡‘Ž›–™‘–‡•–…ƒ•‡•”‡“—‹”‡†ǡ™‡ƒ›…Š‘‘•‡–‘…‘„‹‡–Š‡‹–‘–Š‡‘‡–‡•–•‡–Ǥ
TS1: TEST CASES 1 and 2.

C.2.2.8 Step 6: Derive Test Procedures (TD6)

‰ƒ‹ǡ•‹…‡–Š‡”‡ƒ”‡‘Ž›–™‘–‡•–…ƒ•‡•ƒ†‘‡–‡•–•‡–ǡ™‡ƒ›…Š‘‘•‡–‘†‡ϐ‹‡‘Ž›‘‡–‡•–’”‘…‡†—”‡Ǥ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

C.2.2.9 Branch Testing Coverage

Using the formula provided in 6.3.2 and the test coverage items derived above:
10
Coverage( branch) = × 100% = 100%
10
Thus, 100% coverage of test coverage items for branch testing has been achieved.

C.2.2.10 Decision Testing Coverage

Using the formula provided in 6.3.3 and the test coverage items derived above:
6
Coverage(decision) = × 100% = 100%
6
Thus, 100% coverage of test coverage items for decision testing has been achieved.

ǤʹǤ͵ ”ƒ…Š‘†‹–‹‘‡•–‹‰ǡ”ƒ…Š‘†‹–‹‘‘„‹ƒ–‹‘‡•–‹‰ƒ†‘†‹ϐ‹‡†
Condition Decision Coverage (MCDC) Testing

C.2.3.1 Introduction

”ƒ…Š ‘†‹–‹‘ ‡•–‹‰ǡ ”ƒ…Š ‘†‹–‹‘ ‘„‹ƒ–‹‘ ‡•–‹‰ǡ ƒ† ‘†‹ϐ‹‡† ‘†‹–‹‘ ‡…‹•‹‘
‘˜‡”ƒ‰‡‡•–‹‰ƒ”‡…Ž‘•‡Ž›”‡Žƒ–‡†ǡƒ•ƒ”‡–Š‡ƒ••‘…‹ƒ–‡†…‘˜‡”ƒ‰‡‡ƒ•—”‡•ǤŠ‡ƒ‹‘ˆ–Š‡•‡–Š”‡‡
test design techniques is to derive a set of test cases that cover the conditions within decisions of
the test item according to a chosen level of coverage. For convenience, these test case design and test
coverage measurement approaches are demonstrated using one example.

ǤʹǤ͵Ǥʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the following fragment of code:

LI$RU%DQG&WKHQ
GRBVRPHWKLQJ
HOVH
GRBVRPHWKLQJBHOVH
HQGLI
110 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Š‡ ‘‘Ž‡ƒ …‘†‹–‹‘• ™‹–Š‹ –Š‡ †‡…‹•‹‘ …‘†‹–‹‘ ƒ”‡ ǡ  ƒ† Ǥ Š‡•‡ ƒ› –Š‡•‡Ž˜‡• „‡
comprised of complex expressions involving relational operators. For example, the Boolean condition A
…‘—Ž†„‡ƒ‡š’”‡••‹‘•—…Šƒ•εαǤ‘™‡˜‡”ǡˆ‘”–Š‡•ƒ‡‘ˆ…Žƒ”‹–›ǡ–Š‡ˆ‘ŽŽ‘™‹‰‡šƒ’Ž‡•”‡‰ƒ”†ǡ
B and C as simple Boolean conditions.

C.2.3.3 Step 1: Identify Feature Sets (TD1)

Since the example for all three test design techniques is based on the same test item (the code fragment
ƒ„‘˜‡Ȍǡ‘‡ˆ‡ƒ–—”‡•‡–…ƒ„‡†‡ϐ‹‡†ˆ‘”†‡‘•–”ƒ–‹‘‹ƒŽŽ–Š”‡‡–‡…Š‹“—‡•ǣ
FS1: condition code fragment

C.2.3.4 Step 2: Derive Test Conditions (TD2)

–Š‡•‡–Š”‡‡–‡•–†‡•‹‰–‡…Š‹“—‡•ǡ‡ƒ…Š†‡…‹•‹‘‹–Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’Š‹•ƒ–‡•–…‘†‹–‹‘Ǥ–Š‹•
example, there is one decision:
TCOND1: $RU%DQG& (for FS1)
This test condition is applicable to Branch Condition Testing, Branch Condition Combination Testing,
ƒ†‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡‡•–‹‰Ǥ

C.2.3.5 Branch Condition Testing

C.2.3.5.1 Step 3: Derive Test Coverage Items (TD3)

Branch condition testing examines the individual conditions within multi-condition decisions, with
the aim that each individual condition and each decision takes on both true and false values. The test
coverage items are the Boolean values (true/false) of the conditions within decisions. In this example,
this technique would require Boolean condition A to be evaluated both TRUE and FALSE, Boolean
condition B to be evaluated both TRUE and FALSE and Boolean condition C to be evaluated both TRUE
and FALSE. Therefore, the test coverage items for this technique are:

TCOVER1: α (for TCOND1)

TCOVER2: α  (for TCOND1)

TCOVER3 α (for TCOND1)

TCOVER4: α  (for TCOND1)

TCOVER5: α (for TCOND1)

TCOVER6: α  (for TCOND1)

TCOVER7: ‘”ȋƒ†Ȍα (for TCOND1)

TCOVER8: ‘”ȋƒ†Ȍα  (for TCOND1)

C.2.3.5.2 Step 4: Derive Test Cases (TD4)

”ƒ…Š…‘†‹–‹‘…‘˜‡”ƒ‰‡–‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡
‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰ǡ†‡–‡”‹‹‰–Š‡‹’—–•–Šƒ–
will execute those sub-paths and the expected result of the test and repeating until all the required
coverage is achieved. In this example, this can be achieved with the following set of test inputs (note
that there are alternative sets of test inputs which will also achieve branch condition coverage):

© ISO/IEC 2015 – All rights reserved 111

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table C.3 — Test cases for Branch Condition Testing

Test Case A B C A or (B and C) Test Coverage Items

1 FALSE FALSE FALSE FALSE TCOVER 2, 4, 6
2 TRUE TRUE TRUE TRUE TCOVER 1, 3, 5

 Š‡–‡•–…ƒ•‡•…‘–ƒ‹‡†‹–Š‡–ƒ„Ž‡ƒ„‘˜‡ƒ”‡‘–Dz…‘’Ž‡–‡dz‹–Šƒ––Š‡›†‘‘–‹…Ž—†‡‡š’‡…–‡†
results.

”ƒ…Š…‘†‹–‹‘…‘˜‡”ƒ‰‡…ƒ‘ˆ–‡„‡ƒ…Š‹‡˜‡†™‹–ŠŒ—•––™‘–‡•–…ƒ•‡•ǡ‹””‡•’‡…–‹˜‡‘ˆ–Š‡—„‡”‘ˆ
actual Boolean conditions comprising the overall condition.

C.2.3.5.3 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡ƒ”‡‘Ž›–™‘–‡•–…ƒ•‡•”‡“—‹”‡†ǡ™‡ƒ›…Š‘‘•‡–‘…‘„‹‡–Š‡‹–‘–Š‡‘‡–‡•–•‡–Ǥ
TS1: TEST CASES 1 and 2.

C.2.3.5.4 Step 6: Derive Test Procedures (TD6)

‰ƒ‹ǡ•‹…‡–Š‡”‡ƒ”‡‘Ž›–™‘–‡•–…ƒ•‡•ƒ†‘‡–‡•–•‡–ǡ™‡ƒ›…Š‘‘•‡–‘†‡ϐ‹‡‘Ž›‘‡–‡•–’”‘…‡†—”‡Ǥ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

C.2.3.5.5 Branch Condition Testing Coverage

Using the formula provided in 6.3.4 and the test coverage items derived above:
8
Coverage( branch _ condition) = × 100% = 100%
8
Thus, 100% coverage of test coverage items for branch condition testing has been achieved.

C.2.3.6 Branch Condition Combination Testing

C.2.3.6.1 Step 3: Derive Test Coverage Items (TD3)

In branch condition combination testing, the test coverage items are the unique combinations of Boolean
values of conditions within decisions. In this example, this technique would require all combinations of
Boolean conditions A, B and C to be evaluated. Therefore, the test coverage items for this technique are:

TCOVER1: α ǡ α ǡ α  (for TCOND1)

TCOVER2: αǡ α ǡ α  (for TCOND1)

TCOVER3: α ǡ αǡ α  (for TCOND1)

TCOVER4: αǡ αǡ α  (for TCOND1)

TCOVER5: α ǡ α ǡ α (for TCOND1)

TCOVER6: αǡ α ǡ α (for TCOND1)

TCOVER7: α ǡ αǡ α (for TCOND1)

TCOVER8: αǡ αǡ α (for TCOND1)

112 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

C.2.3.6.2 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ ‡š‡…—–‡† †—”‹‰ –‡•–‹‰ǡ †‡–‡”‹‹‰ –Š‡ ‹’—–• –Šƒ– ™‹ŽŽ ‡š‡…—–‡ –Š‘•‡ •—„Ǧ
paths and the expected result of the test and repeating until the required coverage is achieved. In this
‡šƒ’Ž‡ǡ–Š‹•…ƒ„‡ƒ…Š‹‡˜‡†„›†‡”‹˜‹‰–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…ƒ•‡•ǣ

Table C.4 — Test cases for Branch Condition Combination Testing

Test Case A B C Test Coverage Items

1 FALSE FALSE FALSE TCOVER1
2 TRUE FALSE FALSE TCOVER2
3 FALSE TRUE FALSE TCOVER3
4 TRUE TRUE FALSE TCOVER4
5 FALSE FALSE TRUE TCOVER5
6 TRUE FALSE TRUE TCOVER6
7 ALSE TRUE TRUE TCOVER7
8 TRUE TRUE TRUE TCOVER8

 Š‡–‡•–…ƒ•‡•…‘–ƒ‹‡†‹–Š‡–ƒ„Ž‡ƒ„‘˜‡ƒ”‡‘–Dz…‘’Ž‡–‡dz‹–Šƒ––Š‡›†‘‘–‹…Ž—†‡‡š’‡…–‡†
results.

”ƒ…Š ‘†‹–‹‘ ‘„‹ƒ–‹‘ ‘˜‡”ƒ‰‡ ‹• ˜‡”› –Š‘”‘—‰Šǡ ”‡“—‹”‹‰ ʹn test cases to achieve 100%
coverage of a condition containing n ‘‘Ž‡ƒ …‘†‹–‹‘•Ǥ Š‹• ”ƒ’‹†Ž› „‡…‘‡• —ƒ…Š‹‡˜ƒ„Ž‡ ˆ‘”
complex conditions.

C.2.3.6.3 Step 5: Assemble Test Sets (TD5)

–ƒ›„‡†‡…‹†‡†–Šƒ–ƒŽŽ–‡•–…ƒ•‡•ˆ‘”–Š‹•–‡…Š‹“—‡ƒ”‡…‘„‹‡†‹–‘–Š‡‘‡–‡•–•‡–ǡƒ•ˆ‘ŽŽ‘™•ǣ
TS1: TEST CASES 1, 2, 3, 4, 5, 6, 7, 8.

C.2.3.6.4 Step 6: Derive Test Procedures (TD6)

‹…‡–Š‡”‡‹•‘Ž›‘‡–‡•–•‡–ǡ™‡ƒ›…Š‘‘•‡–‘†‡ϐ‹‡‘‡…‘””‡•’‘†‹‰–‡•–’”‘…‡†—”‡ǡƒ•ˆ‘ŽŽ‘™•ǣ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‡š‡…—–‡†‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

C.2.3.6.5 Branch Condition Combination Testing Coverage

Using the formula provided in 6.3.5 and the test coverage items derived above:
8
Coverage( branch _ condition _ combination) = × 100% = 100%
8
Thus, 100% coverage of test coverage items for branch condition combination testing has been achieved.

ǤʹǤ͵Ǥ͹ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡‡•–‹‰

‘†‹ϐ‹‡† ‘†‹–‹‘ ‡…‹•‹‘ ‘˜‡”ƒ‰‡ ȋȌ ‡•–‹‰ ‹• ƒ ’”ƒ‰ƒ–‹… …‘’”‘‹•‡ ™Š‹…Š ”‡“—‹”‡•
ˆ‡™‡”–‡•–…ƒ•‡•–Šƒ”ƒ…Š‘†‹–‹‘‘„‹ƒ–‹‘‘˜‡”ƒ‰‡Ǥ–‹•™‹†‡Ž›—•‡†‹–Š‡†‡˜‡Ž‘’‡–‘ˆ
ƒ˜‹‘‹…••‘ˆ–™ƒ”‡ǡƒ•”‡“—‹”‡†„›ȀǦͳ͹ͺǤ‡•–‹‰”‡“—‹”‡•–‡•–…ƒ•‡•–‘•Š‘™–Šƒ–‡ƒ…Š
‘‘Ž‡ƒ…‘†‹–‹‘ȋǡƒ†Ȍ…ƒ‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡†‡…‹•‹‘ǤŠ‹•‹•Ž‡••–ŠƒƒŽŽ
–Š‡…‘„‹ƒ–‹‘•ȋƒ•”‡“—‹”‡†„›”ƒ…Š‘†‹–‹‘‘„‹ƒ–‹‘‘˜‡”ƒ‰‡ȌǤ

© ISO/IEC 2015 – All rights reserved 113

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

C.2.3.7.1 Step 3: Derive Test Coverage Items (TD3)

 ‘†‹ϐ‹‡† …‘†‹–‹‘ †‡…‹•‹‘ …‘˜‡”ƒ‰‡ ȋȌ –‡•–‹‰ǡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ –Š‡ —‹“—‡
combinations of individual Boolean values of conditions within decisions that allow a single Boolean
…‘†‹–‹‘–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡Ǥ
‘” –Š‡ ‡šƒ’Ž‡ †‡…‹•‹‘ …‘†‹–‹‘ ȏ ‘” ȋ ƒ† ȌȐǡ –Š‹• Ž‡ƒ†• –‘ ƒ ’ƒ‹” ‘ˆ –‡•– …‘˜‡”ƒ‰‡ ‹–‡•ǡ
where changing the state of A will change the outcome, but B and C remain constant, i.e. that A can
‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡…‘†‹–‹‘ǡƒ•ˆ‘ŽŽ‘™•ǣ

TCOVER1: α ǡ α ǡ α α  (for TCOND1)

TCOVER2: αǡ α ǡ α α (for TCOND1)

‹‹Žƒ”Ž›ˆ‘”ǡ™‡”‡“—‹”‡ƒ’ƒ‹”‘ˆ–‡•–…ƒ•‡•™Š‹…Š•Š‘™–Šƒ–…ƒ‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡ǡ
with A and C remaining constant:

TCOVER3: α ǡ α ǡ α α  (for TCOND1)

TCOVER4: α ǡ αǡ α α (for TCOND1)

‹ƒŽŽ›ˆ‘”™‡”‡“—‹”‡ƒ’ƒ‹”‘ˆ–‡•–…ƒ•‡•™Š‹…Š•Š‘™–Šƒ–…ƒ‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡ǡ
with A and B remaining constant:

TCOVER5: α ǡ αǡ α  α  (for TCOND1)

TCOVER6: α ǡ αǡ α α (for TCOND1)

ƒ˜‹‰…”‡ƒ–‡†–Š‡•‡’ƒ‹”•‘ˆ–‡•–…‘†‹–‹‘•ˆ‘”‡ƒ…Š†‡…‹•‹‘…‘†‹–‹‘•‡’ƒ”ƒ–‡Ž›ǡ‹–…ƒ„‡•‡‡–Šƒ–
TCOVER1 and TCOVER3 are the same, and that TCOVER4 and TCOVER6 are the same. Therefore, the
duplicate test conditions TCOVER3 and TCOVER6 will not be used as a basis for deriving test cases in
the next step.

C.2.3.7.2 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ ‡š‡…—–‡† †—”‹‰ –‡•–‹‰ǡ †‡–‡”‹‹‰ –Š‡ ‹’—–• –Šƒ– ™‹ŽŽ ‡š‡…—–‡ –Š‘•‡ •—„Ǧ
paths and the expected result of the test and repeating until all the required coverage is achieved. In
this example, this can be achieved with the following set of test cases:
 Š‡–‡•–…ƒ•‡•…‘–ƒ‹‡†‹–Š‡–ƒ„Ž‡„‡Ž‘™ƒ”‡‘–Dz…‘’Ž‡–‡dz‹–Šƒ––Š‡›†‘‘–‹…Ž—†‡‡š’‡…–‡†
results.

Table C.5 — Overall set of test cases

Test Case A B C Expected Result Test Coverage Items

1 FALSE FALSE TRUE FALSE TCOVER1, TCOVER3
2 TRUE FALSE TRUE TRUE TCOVER2
3 FALSE TRUE TRUE TRUE TCOVER4, TCOVER6
4 FALSE TRUE FALSE FALSE TCOVER5

•—ƒ”›ǣ
Ȅ ‹••Š‘™–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡†‡…‹•‹‘…‘†‹–‹‘„›–‡•–…ƒ•‡•ͳƒ†ʹǢ
Ȅ ‹••Š‘™–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡†‡…‹•‹‘…‘†‹–‹‘„›–‡•–…ƒ•‡•ͳƒ†͵Ǣƒ†
Ȅ ‹••Š‘™–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡†‡…‹•‹‘…‘†‹–‹‘„›–‡•–…ƒ•‡•͵ƒ†ͶǤ

114 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

‘–‡–Šƒ––Š‡”‡ƒ›„‡ƒŽ–‡”ƒ–‹˜‡•‘Ž—–‹‘•–‘ƒ…Š‹‡˜‹‰Ǥ ‘”‡šƒ’Ž‡ǡ…‘—Ž†Šƒ˜‡„‡‡•Š‘™
–‘‹†‡’‡†‡–Ž›ƒˆˆ‡…––Š‡‘—–…‘‡‘ˆ–Š‡…‘†‹–‹‘„›–Š‡ˆ‘ŽŽ‘™‹‰’ƒ‹”‘ˆ–‡•–…ƒ•‡•ǣ

Table C.6 — Alternative MCDC test cases

Case A B C Outcome
X FALSE TRUE FALSE FALSE
Y TRUE TRUE FALSE TRUE

‡•–…ƒ•‡‹•–Š‡•ƒ‡ƒ•–‡•–…ƒ•‡Ͷƒ„‘˜‡ǡ„—––‡•–…ƒ•‡‹•‘‡™Š‹…ŠŠƒ•‘–„‡‡’”‡˜‹‘—•Ž›—•‡†Ǥ
‘™‡˜‡”ǡƒ•Šƒ•ƒŽ”‡ƒ†›„‡‡ƒ…Š‹‡˜‡†ǡ–‡•–…ƒ•‡‹•‘–”‡“—‹”‡†ˆ‘”…‘˜‡”ƒ‰‡’—”’‘•‡•Ǥ
‘ƒ…Š‹‡˜‡ͳͲͲΨ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡”‡“—‹”‡•ƒ‹‹—‘ˆΪͳ–‡•–…ƒ•‡•ǡƒ†ƒ
maximum of 2n test cases, where n is the number of Boolean conditions within the decision condition. In
contrast, Branch Condition Combination Coverage requires 2n–‡•–…ƒ•‡•Ǥ‹•–Š‡”‡ˆ‘”‡ƒ’”ƒ…–‹…ƒŽ
low-risk compromise with Branch Condition Combination Coverage where condition expressions
‹˜‘Ž˜‡‘”‡–ŠƒŒ—•–ƒˆ‡™‘‘Ž‡ƒ…‘†‹–‹‘•Ǥ

C.2.3.7.3 Step 5: Assemble Test Sets (TD5)

–ƒ›„‡ƒ••—‡†–Šƒ–ƒŽŽ–‡•–…ƒ•‡•†‡ϐ‹‡†‹Table C.5 for this technique are combined into the one
test set, as follows:
TS1: TEST CASES 1, 2, 3, 4.

C.2.3.7.4 Step 6: Derive Test Procedures (TD6)

‹…‡–Š‡”‡‹•‘Ž›‘‡–‡•–•‡–ǡ™‡ƒ›…Š‘‘•‡–‘†‡ϐ‹‡‘‡…‘””‡•’‘†‹‰–‡•–’”‘…‡†—”‡ǡƒ•ˆ‘ŽŽ‘™•ǣ
 ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

ǤʹǤ͵Ǥ͹Ǥͷ ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡‡•–‹‰‘˜‡”ƒ‰‡

Using the formula provided in 6.3.6 and the test coverage items derived above:
4
Coverage(mod ified_condition_decision _ cov erage ) = × 100% = 100%
%
4
Š—•ǡͳͲͲΨ…‘˜‡”ƒ‰‡‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”–‡•–‹‰Šƒ•„‡‡ƒ…Š‹‡˜‡†Ǥ

C.2.3.8 Other Boolean Expressions

One weakness of these three test design techniques and test coverage measurement approaches is that
–Š‡›ƒ”‡˜—Ž‡”ƒ„Ž‡–‘–Š‡’Žƒ…‡‡–‘ˆ‘‘Ž‡ƒ‡š’”‡••‹‘•™Š‡…‘–”‘Ž†‡…‹•‹‘•ƒ”‡’Žƒ…‡†‘—–•‹†‡
of the actual decision condition. For example:
)/$* $RU%DQG&
LI)/$*WKHQ
GRBVRPHWKLQJ
HOVH
GRBVRPHWKLQJBHOVH
HQGLI

‘…‘„ƒ––Š‹•˜—Ž‡”ƒ„‹Ž‹–›ǡƒ’”ƒ…–‹…ƒŽ˜ƒ”‹ƒ–‹‘‘ˆ–Š‡•‡–Š”‡‡–‡•–†‡•‹‰–‡…Š‹“—‡•ƒ†…‘˜‡”ƒ‰‡
‡ƒ•—”‡• ‹• –‘ †‡•‹‰ –‡•–• ˆ‘” ƒŽŽ ‘‘Ž‡ƒ ‡š’”‡••‹‘•ǡ ‘– Œ—•– –Š‘•‡ —•‡† †‹”‡…–Ž› ‹ …‘–”‘Ž ϐŽ‘™
decisions.

© ISO/IEC 2015 – All rights reserved 115

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

C.2.3.9 Optimised Expressions

‘‡ ’”‘‰”ƒ‹‰ Žƒ‰—ƒ‰‡• ƒ† …‘’‹Ž‡”• Dz•Š‘”– …‹”…—‹–dz –Š‡ ‡˜ƒŽ—ƒ–‹‘ ‘ˆ ‘‘Ž‡ƒ ‘’‡”ƒ–‘”• „›
‹‰‘”‹‰ƒ›’ƒ”–‘ˆƒ‡š’”‡••‹‘–Šƒ–†‘‡•‘–Šƒ˜‡ƒ†‹”‡…–‹’ƒ…–‘–Š‡‘—–…‘‡‘ˆ–Šƒ–‡š’”‡••‹‘Ǥ
‘” ‡šƒ’Ž‡ǡ –Š‡  ƒ† ΪΪ Žƒ‰—ƒ‰‡• ƒŽ™ƒ›• •Š‘”– …‹”…—‹– –Š‡ ‘‘Ž‡ƒ Dzƒ†dz ȋƬƬȌ ƒ† Dz‘”dz ȋȁȁȌ
operators, and the Ada programming language provides special short circuit operators and then and
or elseǤ‹–Š–Š‡•‡‡šƒ’Ž‡•ǡ™Š‡–Š‡‘—–…‘‡‘ˆƒ‘‘Ž‡ƒ‘’‡”ƒ–‘”…ƒ„‡†‡–‡”‹‡†ˆ”‘–Š‡ϐ‹”•–
condition, then the second condition will not be evaluated.
The consequence is that it will be infeasible to show coverage of one value of the second condition. For a
short circuited “and” operator, the feasible combinations are True:True, True:False and False:X, where X is
unknown. For a short circuited “or” operator, feasible combinations are False:False, False:True and True:X.
–Š‡”Žƒ‰—ƒ‰‡•ƒ†…‘’‹Ž‡”•ƒ›•Š‘”–…‹”…—‹––Š‡‡˜ƒŽ—ƒ–‹‘‘ˆ‘‘Ž‡ƒ‘’‡”ƒ–‘”•‹ƒ›‘”†‡”Ǥ
this case, the feasible combinations are not known. The degree of short circuit optimisation of Boolean
‘’‡”ƒ–‘”•ƒ›†‡’‡†—’‘…‘’‹Ž‡”•™‹–…Š‡•‘”ƒ›„‡‘—–•‹†‡–Š‡—•‡”ǯ•…‘–”‘ŽǤ
Š‘”–…‹”…—‹–‡†…‘–”‘Žˆ‘”•’”‡•‡–‘‘„•–ƒ…Ž‡–‘”ƒ…Š‘†‹–‹‘‘˜‡”ƒ‰‡‘”‘†‹ϐ‹‡†‘†‹–‹‘
‡…‹•‹‘ ‘˜‡”ƒ‰‡ǡ „—– –Š‡› †‘ ‘„•–”—…– ‡ƒ•—”‡‡– ‘ˆ ”ƒ…Š ‘†‹–‹‘ ‘„‹ƒ–‹‘ ‘˜‡”ƒ‰‡Ǥ
There are situations where it is possible to design test cases which should achieve 100% coverage (from
ƒ–Š‡‘”‡–‹…ƒŽ’‘‹–‘ˆ˜‹‡™Ȍǡ„—–™Š‡”‡‹–‹•‘–’‘••‹„Ž‡–‘ƒ…–—ƒŽŽ›‡ƒ•—”‡–Šƒ–ͳͲͲΨ…‘˜‡”ƒ‰‡Šƒ•
been achieved.

C.2.3.10 Other Branches and Decisions

Š‡ƒ„‘˜‡†‡•…”‹’–‹‘•‘ˆ„”ƒ…Š…‘†‹–‹‘–‡•–‹‰ǡ„”ƒ…Š…‘†‹–‹‘…‘„‹ƒ–‹‘–‡•–‹‰ƒ†
testing and their corresponding coverage measures are given in terms of branches or decisions which
ƒ”‡ …‘–”‘ŽŽ‡† „› ‘‘Ž‡ƒ …‘†‹–‹‘•Ǥ –Š‡” „”ƒ…Š‡• ƒ† †‡…‹•‹‘•ǡ •—…Š ƒ• —Ž–‹Ǧ™ƒ› „”ƒ…Š‡•
ȋ‹’Ž‡‡–‡† „› Dz…ƒ•‡dzǡ Dz•™‹–…Šdz ‘” Dz…‘’—–‡† ‰‘–‘dz •–ƒ–‡‡–•Ȍǡ ƒ† …‘—–‹‰ Ž‘‘’• ȋ‹’Ž‡‡–‡†
„› Dzˆ‘”dz ‘” Dz†‘dz Ž‘‘’• ™‹–Š‘—– ƒ› …‘†‹–‹‘•Ȍ †‘ ‘– —•‡ ‘‘Ž‡ƒ …‘†‹–‹‘•ǡ ƒ† ƒ”‡ –Š‡”‡ˆ‘”‡ ‘–
ƒ††”‡••‡†„›–Š‡†‡•…”‹’–‹‘•Ǥ
‡™ƒ›‘ˆŠƒ†Ž‹‰–Š‹••…‡ƒ”‹‘‹•–‘—•‡‘‡‘ˆ–Š‡•‡–Š”‡‡–‡•–†‡•‹‰–‡…Š‹“—‡•ƒ†‹–•ƒ••‘…‹ƒ–‡†
coverage measure as a supplement to branch testing and branch or decision coverage. Branch testing
™‹ŽŽƒ††”‡••ƒŽŽ•‹’Ž‡†‡…‹•‹‘•ǡ—Ž–‹Ǧ™ƒ›†‡…‹•‹‘•ǡƒ†ƒŽŽŽ‘‘’•Ǥ‘†‹–‹‘–‡•–‹‰™‹ŽŽ–Š‡ƒ††”‡••
the decisions which include Boolean conditions.
’”ƒ…–‹…‡ǡ–‡•–…ƒ•‡•–Šƒ–ƒ…Š‹‡˜‡ͳͲͲΨ…‘˜‡”ƒ‰‡„›‘‡‘ˆ–Š‡•‡‘’–‹‘•™‹ŽŽƒŽ•‘ƒ…Š‹‡˜‡ͳͲͲΨ…‘˜‡”ƒ‰‡
„›–Š‡‘–Š‡”‘’–‹‘Ǥ‘™‡˜‡”ǡŽ‘™‡”Ž‡˜‡Ž•‘ˆ…‘˜‡”ƒ‰‡…ƒ‘–„‡…‘’ƒ”‡†„‡–™‡‡–Š‡–™‘‘’–‹‘•Ǥ

C.2.4 Data Flow Testing

C.2.4.1 Introduction

Š‡ ƒ‹ ‘ˆ †ƒ–ƒ ϐŽ‘™ –‡•–‹‰ ‹• –‘ †‡”‹˜‡ ƒ •‡– ‘ˆ –‡•– …ƒ•‡• –Šƒ– …‘˜‡” –Š‡ ’ƒ–Š• „‡–™‡‡ †‡ϐ‹‹–‹‘•
ƒ† —•‡• ‘ˆ ˜ƒ”‹ƒ„Ž‡• ‹ ƒ –‡•– ‹–‡ ƒ……‘”†‹‰ –‘ ƒ …Š‘•‡ Ž‡˜‡Ž ‘ˆ †‡ϐ‹‹–‹‘Ǧ—•‡ …‘˜‡”ƒ‰‡Ǥ ƒ–ƒ ϐŽ‘™
testing is a structure-based test design technique which aims to execute sub-paths from points where
‡ƒ…Š˜ƒ”‹ƒ„Ž‡‹ƒ–‡•–‹–‡‹•†‡ϐ‹‡†–‘’‘‹–•™Š‡”‡‹–‹•”‡ˆ‡”‡…‡†ǤŠ‡•‡•—„Ǧ’ƒ–Š•ƒ”‡‘™ƒ•
†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•ǤŠ‡†‹ˆˆ‡”‡–†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡…”‹–‡”‹ƒ”‡“—‹”‡†‹ˆˆ‡”‡–†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•ƒ†
sub-paths to be executed. Test sets are generated here to achieve 100% coverage (where possible) for
each of those criteria.
 ƒ–ƒϐŽ‘™–‡•–‹‰‡‡†•–‘†‡ϐ‹‡–Š‡†ƒ–ƒ‘„Œ‡…–•…‘•‹†‡”‡†Ǥ‘‘Ž•™‹ŽŽ–›’‹…ƒŽŽ›”‡‰ƒ”†ƒƒ””ƒ›‘”
”‡…‘”†ƒ•ƒ•‹‰Ž‡†ƒ–ƒ‹–‡”ƒ–Š‡”–Šƒƒ•ƒ…‘’‘•‹–‡‹–‡™‹–Šƒ›…‘•–‹–—‡–•Ǥ‰‘”‹‰–Š‡…‘•–‹–—‡–•‘ˆ
…‘’‘•‹–‡‘„Œ‡…–•”‡†—…‡•–Š‡‡ˆˆ‡…–‹˜‡‡••‘ˆ†ƒ–ƒϐŽ‘™–‡•–‹‰Ǥ

ǤʹǤͶǤʹ ’‡…‹ϐ‹…ƒ–‹‘

‘•‹†‡”–Š‡†ƒ–ƒϐŽ‘™–‡•–‹‰‘ˆ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–‹–‡‹–Š‡†ƒ’”‘‰”ƒ‹‰Žƒ‰—ƒ‰‡ǣ
116 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

SURFHGXUH6ROYHB4XDGUDWLF$%&LQ)ORDW,VB&RPSOH[RXW%RROHDQ55RXW)ORDW
LV
,VB&RPSOH[LVWUXHLIWKHURRWVDUHQRWUHDO
,IWKHWZRURRWVDUHUHDOWKH\DUHSURGXFHGLQ55
'LVFULP)ORDW % % $ &
55)ORDW
EHJLQ
LI'LVFULPWKHQ
,VB&RPSOH[ WUXH
HOVH
,VB&RPSOH[ IDOVH
HQGLI
LIQRW,VB&RPSOH[WKHQ
5 %6TUW'LVFULP $
5 %6TUW'LVFULP $
HQGLI
HQG6ROYHB4XDGUDWLF

‘–‡–Šƒ––Š‡•‡…‘†Ž‹‡‹•‘–ƒ†‡ϐ‹‹–‹‘ȋ‘ˆͳƒ†ʹȌ„—–ƒ†‡…Žƒ”ƒ–‹‘Ǥȋ ‘”Žƒ‰—ƒ‰‡•™‹–Š†‡ˆƒ—Ž–
‹‹–‹ƒŽ‹•ƒ–‹‘ǡ‹–™‘—Ž†„‡ƒ†‡ϐ‹‹–‹‘ǤȌ

C.2.4.3 Step 1: Identify Feature Sets (TD1)

‹…‡–Š‡”‡‹•‘Ž›‘‡ˆ—…–‹‘—†‡”–‡•–ǡ–Š‡”‡‹•‘‡ˆ‡ƒ–—”‡•‡––Šƒ–…ƒ„‡†‡ϐ‹‡†ǣ
ͳǣ‘Ž˜‡̴—ƒ†”ƒ–‹…ˆ—…–‹‘

C.2.4.4 Step 2: Derive Test Conditions (TD2)

Š‡ϐ‹”•–•–‡’‹•–‘Ž‹•––Š‡˜ƒ”‹ƒ„Ž‡•—•‡†‹–Š‡–‡•–‹–‡ǤŠ‡•‡ƒ”‡ǣǡǡǡ‹•…”‹ǡ•̴‘’Ž‡šǡͳƒ†
R2. Next, each occurrence of a variable in the test item is cross referenced against the program listing
ƒ†ƒ••‹‰‡†ƒ…ƒ–‡‰‘”›ȋ†‡ϐ‹‹–‹‘ǡ’”‡†‹…ƒ–‡Ǧ—•‡‘”…‘’—–ƒ–‹‘Ǧ—•‡ȌǤ

Table C.7 — Occurrence of variables and their categories

Category
Line †‡ϐ‹‹–‹‘ c-use p-use
0 A, B, C
1 Discrim A, B, C
2
3
4 Discrim
5 •̴‘’Ž‡š
6
7 •̴‘’Ž‡š
8
9 •̴‘’Ž‡š
10 R1 A, B, Discrim
11 R2 A, B, Discrim
12
13 ͳǡʹǡ•̴‘’Ž‡š

Š‡–‡•–…‘†‹–‹‘•ƒ”‡–Š‡†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•Ǥ

© ISO/IEC 2015 – All rights reserved 117

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Š‡‡š–•–‡’‹•–‘‹†‡–‹ˆ›–Š‡†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•ƒ†–Š‡‹”–›’‡ȋ…Ǧ—•‡‘”’Ǧ—•‡Ȍǡ‡ƒ…Š‘ˆ™Š‹…Šƒ”‡
–‡•–…‘˜‡”ƒ‰‡‹–‡•ǡ„›‹†‡–‹ˆ›‹‰Ž‹•ˆ”‘‡ƒ…Š‡–”›‹–Š‡†‡ϐ‹‹–‹‘…‘Ž—–‘‡ƒ…ŠŽƒ–‡”‡–”›ˆ‘”
that variable in the c-use or p-use column.

ƒ„Ž‡ǤͺȄ†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•ƒ†–Š‡‹”–›’‡

‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹” Variables Test

ȋ•–ƒ”–Ž‹‡಩‡†Ž‹‡Ȍ c-use p-use Conditions
Ͳ՜ͳ A TCOND1
B TCOND2
C TCOND3
Ͳ՜ͳͲ A TCOND4
B TCOND5
Ͳ՜ͳͳ A TCOND6
B TCOND7
ͳ՜Ͷ Discrim TCOND8
ͳ՜ͳͲ Discrim TCOND9
ͳ՜ͳͳ Discrim TCOND10
ͷ՜ͻ •̴‘’Ž‡š TCOND11
͹՜ͻ •̴‘’Ž‡š TCOND12
ͳͲ՜ͳ͵ R1 TCOND13
ͳͳ՜ͳ͵ R2 TCOND14
ͷ՜ͳ͵ •̴‘’Ž‡š TCOND15
͹՜ͳ͵ •̴‘’Ž‡š TCOND16

‘–‡–Šƒ–‹–‹•‘–ƒŽ™ƒ›•‡…‡••ƒ”›–‘†‡”‹˜‡ƒŽŽ‘ˆ–Š‡†‡ˆǦ—•‡’ƒ‹”•ȋƒ•Šƒ•„‡‡†‘‡Š‡”‡Ȍ‹‘”†‡”–‘
proceed with deriving the test coverage items (depending on the technique being used).

ǤʹǤͶǤͷ ŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

ǤʹǤͶǤͷǤͳ –‡’͵ƒǣ‡”‹˜‡‡•–‘˜‡”ƒ‰‡–‡•ȋ͵ȌȂŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

 ƒŽŽǦ†‡ϐ‹‹–‹‘• –‡•–‹‰ǡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ –Š‡ …‘–”‘Ž ϐŽ‘™ •—„Ǧ’ƒ–Š• ˆ”‘ ƒ ˜ƒ”‹ƒ„Ž‡
†‡ϐ‹‹–‹‘–‘•‘‡—•‡ȋ‡‹–Š‡”’Ǧ—•‡‘”…Ǧ—•‡Ȍ‘ˆ–Šƒ–†‡ϐ‹‹–‹‘Ǥ
The following table shows one set of def-use pairs that meet this criterion.

ƒ„Ž‡ǤͻȄŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

ŽŽǦ‡ϐ‹‹–‹‘•
Test Coverage
Items Variables †‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹” Test Conditions
TCOVER1 A Ͳ՜ͳ TCOND1
TCOVER2 B Ͳ՜ͳ TCOND2
TCOVER3 C Ͳ՜ͳ TCOND3
TCOVER4 Discrim ͳ՜Ͷ TCOND8
TCOVER5 •̴‘’Ž‡š ͷ՜ͻ TCOND11
TCOVER6 •̴‘’Ž‡š ͹՜ͻ TCOND12
TCOVER7 R1 ͳͲ՜ͳ͵ TCOND13
TCOVER8 R2 ͳͳ՜ͳ͵ TCOND14
118 © ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

ǤʹǤͶǤͷǤʹ –‡’Ͷƒǣ‡”‹˜‡‡•–ƒ•‡•ȋͶȌȂŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ ‡š‡…—–‡† †—”‹‰ –‡•–‹‰ǡ †‡–‡”‹‹‰ –Š‡ ‹’—–• –Šƒ– ™‹ŽŽ ‡š‡…—–‡ –Š‘•‡ •—„Ǧ
paths and the expected result of the test and repeating until the required coverage is achieved. To
ƒ…Š‹‡˜‡ͳͲͲΨƒŽŽǦ†‡ϐ‹‹–‹‘•†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡ƒ–Ž‡ƒ•–‘‡•—„Ǧ’ƒ–Šˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘
•‘‡—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘ȋ‡‹–Š‡”’Ǧ—•‡‘”…Ǧ—•‡Ȍ—•–„‡‡š‡…—–‡†ǤŠ‡ˆ‘ŽŽ‘™‹‰–‡•–•‡–™‘—Ž†•ƒ–‹•ˆ›
this requirement:

ƒ„Ž‡ǤͳͲȄ‡•–ƒ•‡•ˆ‘”ŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰

ŽŽǦ‡ϐ‹‹–‹‘• INPUTS EXPECTED RESULT

‡ϐ‹‹-
Test tion-use Sub- Test Coverage
Case Variables pairs paths Items A B C Is_Complex R1 R2
1 •̴‘’Ž‡š ͹՜ͻ 7-8-9 TCOVER6 1 2 1 FALSE -1 -1
R1 ͳͲ՜ͳ͵ 10-11-12- TCOVER7
13
R2 ͳͳ՜ͳ͵ 11-12-13 TCOVER8
2 A,B,C, Ͳ՜ͳ 0-1 TCOVER1, 1 1 1 TRUE unass. unass.
TCOVER2,
TCOVER3
Discrim ͳ՜Ͷ 1-2-3-4 TCOVER4
•̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOVER5

ǤʹǤͶǤͷǤ͵ ŽŽǦ‡ϐ‹‹–‹‘•‡•–‹‰‘˜‡”ƒ‰‡

Using the formula provided in 6.3.7.1 and the test coverage items derived above:
8
Coverage(all _ definitions ) = × 100% = 100%
8
Š—•ǡͳͲͲΨ…‘˜‡”ƒ‰‡‘ˆ–‡•–…‘˜‡”ƒ‰‡‹–‡•ˆ‘”ƒŽŽǦ†‡ϐ‹‹–‹‘•–‡•–‹‰Šƒ•„‡‡ƒ…Š‹‡˜‡†Ǥ

C.2.4.6 All-C-Uses Testing

C.2.4.6.1 Step 3b: Derive Test Coverage Items (TD3) – All-C-Uses Testing

ƒŽŽǦ…Ǧ—•‡•–‡•–‹‰ǡ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•ˆ”‘ƒ˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘
‡˜‡”›…Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘ǤŠ‡ˆ‘ŽŽ‘™‹‰–ƒ„Ž‡•Š‘™•‘‡•‡–‘ˆ†‡ˆǦ—•‡’ƒ‹”•–Šƒ–‡‡––Š‹•…”‹–‡”‹‘Ǥ

Table C.11 — All-C-Uses Testing

All-C-Uses
Test Coverage ‡ϐ‹‹–‹‘Ǧ—•‡
Items Variable pair Sub-path Test Conditions
TCOVER1 A Ͳ՜ͳ 0-1 TCOND1
TCOVER2 B Ͳ՜ͳ 0-1 TCOND2
TCOVER3 C Ͳ՜ͳ 0-1 TCOND3
TCOVER4 A Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND4
TCOVER5 B Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND5
TCOVER6 A Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND6
TCOVER7 B Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND7

© ISO/IEC 2015 – All rights reserved 119

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table C.11 (continued)

All-C-Uses
Test Coverage ‡ϐ‹‹–‹‘Ǧ—•‡
Items Variable pair Sub-path Test Conditions
TCOVER8 Discrim ͳ՜ͳͲ 1-2-3-4-6-7-8-9-10 TCOND9
TCOVER9 Discrim ͳ՜ͳͳ 1-2-3-4-6-7-8-9-10-11 TCOND10
TCOVER10 R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOND13
TCOVER11 R2 ͳͳ՜ͳ͵ 11-12-13 TCOND14
TCOVER12 •̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOND15
TCOVER13 •̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11-12-13 TCOND16

C.2.4.6.2 Step 4b: Derive Test Cases (TD4) – All-C-Uses Testing

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ ‡š‡…—–‡† †—”‹‰ –‡•–‹‰ǡ †‡–‡”‹‹‰ –Š‡ ‹’—–• –Šƒ– ™‹ŽŽ ‡š‡…—–‡ –Š‘•‡ •—„Ǧ
paths and the expected result of the test and repeating until the required coverage is achieved. To
ƒ…Š‹‡˜‡ͳͲͲΨƒŽŽǦ…Ǧ—•‡•†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡ƒ–Ž‡ƒ•–‘‡•—„Ǧ’ƒ–Šˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡˜‡”›
…Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘—•–„‡‡š‡…—–‡†Ǥ

Table C.12 — Test Cases for All-C-Uses Testing

All-C-uses INPUTS EXPECTED RESULT

‡ϐ‹‹-
Test tion-use Test Coverage
Case Variables pairs Sub-paths Items A B C Is_Complex R1 R2
1 A, B, C Ͳ՜ͳ 0-1 TCOVER1, 1 2 1 FALSE -1 -1
TCOVER2,
TCOVER3
A, B Ͳ՜ͳͲǡ 0-1-2-3-4-6- TCOVER4,
7-8-9-10 TCOVER5
A, B Ͳ՜ͳͳ 0-1-2-3-4-6- TCOVER6,
7-8-9-10-11 TCOVER7
Discrim ͳ՜ͳͲ 1-2-3-4-6-7- TCOVER8
8-9-10
ͳ՜ͳͳ 1-2-3-4-6-7- TCOVER9
8-9-10-11
R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOVER10
R2 ͳͳ՜ͳ͵ 11-12-13 TCOVER11
•̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11- TCOVER13
12-13
2 •̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOVER12 1 1 1 TRUE unass. unass.

C.2.4.6.3 All-C-Uses Testing Coverage

Using the formula provided in 6.3.7.2 and the test coverage items derived above:
13
Coverage(all −c −uses ) = × 100% = 100%
13
Thus, 100% coverage of test coverage items for all-c-uses testing has been achieved.

120 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

C.2.4.7 All-P-Uses Testing

C.2.4.7.1 Step 3c: Derive Test Coverage Items (TD3) – All-P-Uses Testing

ƒŽŽǦ’Ǧ—•‡•–‡•–‹‰ǡ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡–Š‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•ˆ”‘ƒ˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘
‡˜‡”›’Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘Ǥ
The following table shows one set of def-use pairs that meet this criterion.

Table C.13 — All-P-Uses Testing

All-P-Uses
Test Coverage Items Variables ‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹” Test Conditions
TCOVER1 Discrim ͳ՜Ͷ TCOND8
TCOVER2 •̴‘’Ž‡š ͷ՜ͻ TCOND11
TCOVER3 •̴‘’Ž‡š ͹՜ͻ TCOND12

C.2.4.7.2 Step 4c: Derive Test Cases (TD4) – All-P-Uses Testing

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ– Šƒ˜‡ ‘– ›‡– „‡‡ ‡š‡…—–‡† †—”‹‰ –‡•–‹‰ǡ †‡–‡”‹‹‰ –Š‡ ‹’—–• –Šƒ– ™‹ŽŽ ‡š‡…—–‡ –Š‘•‡ •—„Ǧ
paths and the expected result of the test and repeating until the required coverage is achieved. To
ƒ…Š‹‡˜‡ͳͲͲΨƒŽŽǦ’Ǧ—•‡•†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡ƒ–Ž‡ƒ•–‘‡•—„Ǧ’ƒ–Šˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡˜‡”›
’Ǧ—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘—•–„‡‡š‡…—–‡†ǤŠ‡ˆ‘ŽŽ‘™‹‰–‡•–•‡–™‘—Ž†•ƒ–‹•ˆ›–Š‹•”‡“—‹”‡‡–ǣ

Table C.14 — Test cases for All-P-Uses Testing

All-P-Uses INPUTS EXPECTED RESULT

‡ϐ‹‹-
Test tion-use Test Coverage
Case Variables pair sub-paths Items A B C Is_Complex R1 R2
1 •̴‘’Ž‡š ͹՜ͻ 7-8-9 TCOVER3 1 2 1 FALSE -1 -1
2 Discrim ͳ՜Ͷ 1-2-3-4 TCOVER1 1 1 1 TRUE unass. unass.
•̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOVER2

C.2.4.7.3 All-P-Uses Testing Coverage

Using the formula provided in 6.3.7.3 and the test coverage items derived above:
3
Coverage(all − p−uses ) = × 100% = 100%
3
Thus, 100% coverage of test coverage items for all-p-uses testing has been achieved.

C.2.4.8 All-Uses Testing

C.2.4.8.1 Step 3d: Derive Test Coverage Items (TD3) – All-Uses Testing

ƒŽŽǦ—•‡•–‡•–‹‰ǡ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•ˆ”‘ƒ˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡˜‡”›
—•‡ȋ„‘–Š’Ǧ—•‡ƒ†…Ǧ—•‡Ȍ‘ˆ–Šƒ–†‡ϐ‹‹–‹‘Ǥ
The following table shows one set of def-use pairs that meet this criterion.

© ISO/IEC 2015 – All rights reserved 121

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table C.15 — All-Uses Testing

All-Uses / All DU-Paths

Test Coverage Items Variables d-u pair Sub-path Test Conditions
TCOVER1 A Ͳ՜ͳ 0-1 TCOND1
TCOVER2 B Ͳ՜ͳ 0-1 TCOND2
TCOVER3 C Ͳ՜ͳ 0-1 TCOND3
TCOVER4 A Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND4
TCOVER5 B Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND5
TCOVER6 A Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND6
TCOVER7 B Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND7
TCOVER8 Discrim ͳ՜Ͷ 1-2-3-4 TCOND8
TCOVER9 Discrim ͳ՜ͳͲ 1-2-3-4-6-7-8-9-10 TCOND9
TCOVER10 Discrim ͳ՜ͳͳ 1-2-3-4-6-7-8-9-10-11 TCOND10
TCOVER11 •̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOND11
TCOVER12 •̴‘’Ž‡š ͹՜ͻ 7-8-9 TCOND12
TCOVER13 R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOND13
TCOVER14 R2 ͳͳ՜ͳ͵ 11-12-13 TCOND14
TCOVER15 •̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOND15
TCOVER16 •̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11-12-13 TCOND16

C.2.4.8.2 Step 4d: Derive Test Cases (TD4) – All-Uses Testing

‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡–‡•–…‘˜‡”ƒ‰‡‹–‡•
–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰ǡ†‡–‡”‹‹‰–Š‡‹’—–•–Šƒ–™‹ŽŽ‡š‡…—–‡–Š‘•‡•—„Ǧ’ƒ–Š•
and the expected result of the test and repeating until the required coverage is achieved. To achieve
ͳͲͲΨƒŽŽǦ—•‡•†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡ƒ–Ž‡ƒ•–‘‡•—„Ǧ’ƒ–Šˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘–‘‡˜‡”›—•‡‘ˆ–Šƒ–
†‡ϐ‹‹–‹‘ȋ„‘–Š’Ǧ—•‡ƒ†…Ǧ—•‡Ȍ—•–„‡‡š‡…—–‡†ǤŠ‡ˆ‘ŽŽ‘™‹‰–‡•–•‡–™‘—Ž†•ƒ–‹•ˆ›–Š‹•”‡“—‹”‡‡–ǣ

122 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Table C.16 — Test Cases for All-Uses Testing

All-Uses INPUTS EXPECTED RESULT

Test Test Coverage
Case Variables d-u pair Sub-paths Items A B C Is_Complex R1 R2
1 A, B, C Ͳ՜ͳ 0-1 TCOVER1, 1 2 1 FALSE -1 -1
TCOVER2,
TCOVER3
A, B Ͳ՜ͳͲ 0-1-2-3-4-6-7- TCOVER4,
8-9-10 TCOVER5
A, B Ͳ՜ͳͳ 0-1-2-3-4-6-7- TCOVER6,
8-9-10-11 TCOVER7
Discrim ͳ՜Ͷ 1-2-3-4 TCOVER8
ͳ՜ͳͲ 1-2-3-4-6-7-8- TCOVER9
9-10
ͳ՜ͳͳ 1-2-3-4-6-7-8- TCOVER10
9-10-11
•̴‘’Ž‡š ͹՜ͻ 7-8-9 TCOVER12
R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOVER13
R2 ͳͳ՜ͳ͵ 11-12-13 TCOVER14
•̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11- TCOVER16
12-13
2 •̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOVER11 1 1 1 TRUE unass. unass.
•̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOVER15

C.2.4.8.3 All-Uses Testing Coverage

Using the formula provided in 6.3.7.4 and the test coverage items derived above:
16
Coverage(all _ uses ) = × 100% = 100%
16
Thus, 100% coverage of test coverage items for all-uses testing has been achieved.

C.2.4.9 All-DU-Paths Testing

C.2.4.9.1 Step 3e: Derive Test Coverage Items (TD3) – All-DU-Paths Testing

‘ƒ…Š‹‡˜‡ͳͲͲΨƒŽŽǦ†—Ǧ’ƒ–Š•†ƒ–ƒϐŽ‘™…‘˜‡”ƒ‰‡‡˜‡”›Dz•‹’Ž‡•—„Ǧ’ƒ–Šdzˆ”‘‡ƒ…Š˜ƒ”‹ƒ„Ž‡†‡ϐ‹‹–‹‘
–‘‡˜‡”›—•‡‘ˆ–Šƒ–†‡ϐ‹‹–‹‘—•–„‡‡š‡…—–‡†ǤŠ‹•†‹ˆˆ‡”•ˆ”‘ƒŽŽǦ—•‡•‹–Šƒ–every simple sub-path
„‡–™‡‡–Š‡†‡ϐ‹‹–‹‘Ǧ—•‡’ƒ‹”•—•–„‡‡š‡…—–‡†Ǥ–ƒϐ‹”•–‰Žƒ…‡ǡ‹–ƒ’’‡ƒ”•ƒ•‹ˆ–Š‡”‡ƒ”‡–™‘•—„Ǧ
’ƒ–Š•–Š”‘—‰Š–Š‡–‡•–‹–‡–Šƒ–ƒ”‡‘–ƒŽ”‡ƒ†›‹†‡–‹ϐ‹‡†‹–Š‡ƒŽŽǦ—•‡•–‡•–…ƒ•‡•ǤŠ‡•‡ƒ”‡ͲǦͳǦͶǦͷǦͻǦͳͲ
and 1-4-5-9-10. However, both of these sub-paths are infeasible (and so no test cases can be generated to
‡š‡”…‹•‡–Š‡ȌǤŠ—•ǡ–Š‡›ƒ”‡‘–…‘•‹†‡”‡†–‘„‡Dz•‹’Ž‡•—„Ǧ’ƒ–Š•dzˆ‘”ƒŽŽǦ†—Ǧ’ƒ–Š•–‡•–‹‰Ǥ

Table C.17 — All-DU-Paths Testing

All DU-Paths
Test Coverage
Items Variables d-u pair Sub-path Test Conditions
TCOVER1 A Ͳ՜ͳ 0-1 TCOND1
TCOVER2 B Ͳ՜ͳ 0-1 TCOND2
TCOVER3 C Ͳ՜ͳ 0-1 TCOND3

© ISO/IEC 2015 – All rights reserved 123

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table C.17 (continued)

All DU-Paths
Test Coverage
Items Variables d-u pair Sub-path Test Conditions
TCOVER4 A Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND4
TCOVER5 B Ͳ՜ͳͲ 0-1-2-3-4-6-7-8-9-10 TCOND5
TCOVER6 A Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND6
TCOVER7 B Ͳ՜ͳͳ 0-1-2-3-4-6-7-8-9-10-11 TCOND7
TCOVER8 Discrim ͳ՜Ͷ 1-2-3-4 TCOND8
TCOVER9 Discrim ͳ՜ͳͲ 1-2-3-4-6-7-8-9-10 TCOND9
TCOVER10 Discrim ͳ՜ͳͳ 1-2-3-4-6-7-8-9-10-11 TCOND10
TCOVER11 •̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOND11
TCOVER12 •̴‘’Ž‡š ͹՜ͻ 7-8-9 TCOND12
TCOVER13 R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOND13
TCOVER14 R2 ͳͳ՜ͳ͵ 11-12-13 TCOND14
TCOVER15 •̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOND15
TCOVER16 •̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11-12-13 TCOND16

C.2.4.9.2 Step 4e: Derive Test Cases (TD4) – All-DU-Paths Testing

Test cases for all-du-paths can now be derived. The same set of test cases that were derived for all-
uses also achieves the maximum level of test coverage item coverage possible for all-du-paths testing
in this example.

Table C.18 — Test Cases for All-DU-Paths Testing

All DU-Paths INPUTS EXPECTED RESULT

Test Test Coverage
Case Variables d-u pair Sub-paths Items A B C Is_Complex R1 R2
1 A, B, C Ͳ՜ͳ 0-1 TCOVER1, 1 2 1 FALSE -1 -1
TCOVER2,
TCOVER3
A, B Ͳ՜ͳͲ 0-1-2-3-4-6-7-8- TCOVER4,
9-10 TCOVER5
Ͳ՜ͳͳ 0-1-2-3-4-6-7-8- TCOVER6,
9-10-11 TCOVER7
Discrim ͳ՜Ͷ 1-2-3-4 TCOVER8
ͳ՜ͳͲ 1-2-3-4-6-7-8- TCOVER9
9-10
•̴‘’Ž‡š ͳ՜ͳͳ 1-2-3-4-6-7-8-9- TCOVER10
10-11
͹՜ͻ 7-8-9 TCOVER12
R1 ͳͲ՜ͳ͵ 10-11-12-13 TCOVER13
R2 ͳͳ՜ͳ͵ 11-12-13 TCOVER14
•̴‘’Ž‡š ͹՜ͳ͵ 7-8-9-10-11-12- TCOVER16
13
2 •̴‘’Ž‡š ͷ՜ͻ 5-8-9 TCOVER11 1 1 1 TRUE unass. unass.
•̴‘’Ž‡š ͷ՜ͳ͵ 5-8-9-12-13 TCOVER15

124 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

C.2.4.9.3 All-DU-Paths Testing Coverage

Using the formula provided in 6.3.7.5 and the test coverage items derived above:
16
Coverage(all −du− paths ) = × 100% = 100%
16
Thus, 100% coverage of test coverage items for all-du-paths testing has been achieved.

C.2.4.9.4 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡–‡•–…ƒ•‡•†‡”‹˜‡†ˆ‘”ƒŽŽǦ—•‡•…‘˜‡”ƒŽŽ–‡•–…‘˜‡”ƒ‰‡‹–‡•ǡ–Š‡›™‹ŽŽ„‡—•‡†ƒ•ƒ‡šƒ’Ž‡
‘ˆ –Š‡ ‹†‡–‹ϐ‹…ƒ–‹‘ ‘ˆ –‡•– •‡–•Ǥ ŽŽ –‡•– …ƒ•‡• –Šƒ– …ƒ—•‡ •̴‘’Ž‡š –‘ …‘’—–‡ –‘  …‘—Ž† „‡
combined into one test set, and all those that compute to TRUE in another test set, as follows:
TS1: TEST CASE 1.
TS2: TEST CASE 2.

C.2.4.9.5 Step 6: Derive Test Procedures (TD6)

All test sets could be combined into the one test procedure to be executed in sequential order, as follows:
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡˆ‘ŽŽ‘™‡†„›–Š‘•‡‹ʹǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–•Ǥ

© ISO/IEC 2015 – All rights reserved 125

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Annex D
(informative)

Guidelines and Examples for the Application of Experience-Based

Test Design Techniques

D.1 Guidelines and Examples for Experience-Based Testing

D.1.1 Overview
This annex provides guidance on the requirements in 5.4 and 6.4. This clause demonstrates the
application of an experience-based test design technique to an example problem. The example follows
–Š‡‡•–‡•‹‰ƒ†’Ž‡‡–ƒ–‹‘”‘…‡••–Šƒ–‹•†‡ϐ‹‡†‹ȀȀʹͻͳͳͻǦʹǤ

D.2 Experience-Based Test Design Technique Examples

D.2.1 Error Guessing

D.2.1.1 Introduction

Š‡ƒ‹‘ˆ‡””‘”‰—‡••‹‰‹•–‘†‡”‹˜‡ƒ•‡–‘ˆ–‡•–…ƒ•‡•–Šƒ–…‘˜‡”Ž‹‡Ž›‡””‘”•ǡ—•‹‰ƒ–‡•–‡”ǯ•‘™Ž‡†‰‡
ƒ†‡š’‡”‹‡…‡™‹–Š’”‡˜‹‘—•–‡•–‹–‡•Ǥ‡•–…ƒ•‡•ƒ”‡†‡”‹˜‡†–‘‡š‡”…‹•‡‡ƒ…Š–›’‡‘ˆ‡””‘”–Šƒ–‹•
‹†‡–‹ϐ‹‡†„›–Š‡–‡•–‡”ƒ•Ž‹‡Ž›„‡‹‰’”‡•‡–‹–Š‡…—””‡––‡•–‹–‡ǤŠ‹•–‡…Š‹“—‡™‘—Ž†–›’‹…ƒŽŽ›
„‡ƒ’’Ž‹‡†ƒˆ–‡”‘–Š‡”•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†–‡•–†‡•‹‰–‡…Š‹“—‡••—…Šƒ•‡“—‹˜ƒŽ‡…‡’ƒ”–‹–‹‘‹‰ƒ†
„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•ǡ–‘•—’’Ž‡‡––Š‡–›’‡•‘ˆ‡””‘”•–ƒ”‰‡–‡†„›–Š‘•‡–‡…Š‹“—‡•Ǥ

ǤʹǤͳǤʹ ’‡…‹ϐ‹…ƒ–‹‘

Consider the example test item, generate_gradingǡ™Š‹…Š™ƒ•—•‡†ƒ•–Š‡‡šƒ’Ž‡ˆ‘”„‘—†ƒ”›˜ƒŽ—‡

ƒƒŽ›•‹•ƒ†™Š‹…ŠŠƒ†–Š‡ˆ‘ŽŽ‘™‹‰–‡•–„ƒ•‹•ǣ
The component receives an exam mark (out of 75) and a coursework (c/w) mark (out of 25) as input, from
which it outputs a grade for the course in the range ‘A’ to ‘D’. The grade is generated by calculating the
overall mark, which is sum of the exam and c/w marks, as follows:

greater than or equal to 70 - ‘A’

greater than or equal to 50, but less than 70 - ‘B’

greater than or equal to 30, but less than 50 - ‘C’

less than 30 - ‘D’

Where invalid input(s) are detected (e.g. a mark is outside its expected range) then a fault message (‘FM’) is
generated. All inputs are passed as integers.

D.2.1.3 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: generate_grading function

126 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

D.2.1.4 Step 2: Derive Test Conditions (TD2)

‡•–…‘†‹–‹‘•ƒ”‡‹†‡–‹ϐ‹‡†„›†‡”‹˜‹‰ƒŽ‹•–‘ˆ’‘–‡–‹ƒŽ–›’‡•‘ˆ‡””‘”•–Šƒ–ƒ›„‡’”‡•‡–‹–Š‡
test item, based on knowledge and experience of similar errors in other test items that were tested in
the past. For the generate_gradingˆ—…–‹‘ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘†‹–‹‘•ƒ›„‡†‡”‹˜‡†ǣ

TCOND1: enter NULL (for FS1)

TCOND2: enter 0 (for FS1)

TCOND3: enter negative number (for FS1)

TCOND4: enter inputs in reverse order (for FS1)

TCOND5: ‡–‡”˜‡”›Žƒ”‰‡—„‡”ȋ‡Ǥ‰ǤͳͲ†‹‰‹–•Ȍ (for FS1)

TCOND6: ‡–‡”˜‡”›Žƒ”‰‡•–”‹‰‘ˆƒŽ’Šƒ•ȋ‡Ǥ‰ǤͳͲƒŽ’Šƒ•Ȍ (for FS1)

D.2.1.5 Step 3: Derive Test Coverage Items (TD3)

ƒ…Š‰‡‡”‹…–›’‡‘ˆ†‡ˆ‡…–‹•ƒ–‡•–…‘˜‡”ƒ‰‡‹–‡ȋ‹Ǥ‡Ǥ–Š‡–‡•–…‘˜‡”ƒ‰‡‹–‡•ƒ”‡–Š‡•ƒ‡ƒ•–Š‡–‡•–
…‘†‹–‹‘•ȌǤŠ‡”‡ˆ‘”‡ǡ–Š‡ˆ‘ŽŽ‘™‹‰–‡•–…‘˜‡”ƒ‰‡‹–‡•…ƒ„‡†‡ϐ‹‡†ǣ

TCOVER1: enter NULL (for TCOND1)

TCOVER2: enter 0 (for TCOND2)

TCOVER3: enter negative number (for TCOND3)

TCOVER4: enter inputs in reverse order (for TCOND4)

TCOVER5: ‡–‡”˜‡”›Žƒ”‰‡—„‡”ȋ‡Ǥ‰ǤͳͲ†‹‰‹–•Ȍ (for TCOND5)

TCOVER6 ‡–‡”˜‡”›Žƒ”‰‡•–”‹‰‘ˆƒŽ’Šƒ•ȋ‡Ǥ‰ǤͳͲƒŽ’Šƒ•Ȍ (for TCOND6)

D.2.1.6 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•…ƒ‘™„‡†‡”‹˜‡†„›•‡Ž‡…–‹‰ƒ†‡ˆ‡…––›’‡ˆ‘”‹…Ž—•‹‘‹–Š‡…—””‡––‡•–…ƒ•‡ǡ‹†‡–‹ˆ›‹‰
‹’—–ȋ•Ȍ–‘‡š‡”…‹•‡–Š‡’ƒ”ƒ‡–‡”•‘ˆ–Š‡–‡•–…ƒ•‡ƒ……‘”†‹‰–‘–Š‡…Š‘•‡†‡ˆ‡…––›’‡ǡ†‡–‡”‹‹‰–Š‡
expected result and repeating until all test coverage items are included in a test case. For this example,
this results in the following test cases.

Table D.1 — Test cases for error guessing

Test Case 1 2 3 4
Input (exam mark) NULL 25 NULL 0
Input (c/w mark) 20 NULL NULL 20
total mark (as calculated) 20 25 NULL 20
Test Coverage Item TCOVER1 exam TCOVER1 TCOVER1 TCOVER2 exam
mark c/w mark ‡šƒƬ…Ȁ™ƒ”• mark
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table D.2 — Test cases for error guessing continued

Test Case 5 6 7 8
Input (exam mark) 25 0 -25 25
Input (c/w mark) 0 0 20 -25

© ISO/IEC 2015 – All rights reserved 127

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table D.2 (continued)

Test Case 5 6 7 8
total mark (as calculated) 25 0 -5 0
Test Coverage Item TCOVER2 TCOVER2 TCOVER3 exam TCOVER3
c/w mark ‡šƒƬ…Ȁ™ƒ”• mark c/w mark
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table D.3 — Test cases for error guessing continued

Test Case 9 10 11 12
Input (exam mark) -25 20 1234567890 25
Input (c/w mark) -50 55 20 1234567890
total mark (as calculated) -75 75 1234567910 1234567915
Test Coverage Item TCOVER3 TCOVER4 TCOVER5 exam TCOVER5
‡šƒƬ…Ȁ™ƒ” ‡šƒƬ…Ȁ™ƒ” mark c/w mark
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

Table D.4 — Test cases for error guessing continued

Test Case 13 14 15 16
Input (exam mark) 1234567890 ƒ„…†‡ˆ‰Š‹Œ 25 ƒ„…†‡ˆ‰Š‹Œ
Input (c/w mark) 1234567890 20 ƒ„…†‡ˆ‰Š‹Œ ƒ„…†‡ˆ‰Š‹Œ
total mark (as calculated) 2469135780 NULL NULL NULL
Test Coverage Item TCOVER5 TCOVER6 exam TCOVER6 TCOVER6
‡šƒƬ…Ȁ™ƒ” mark c/w mark ‡šƒƬ…Ȁ™ƒ”
Exp. Output Ǯ ǯ Ǯ ǯ Ǯ ǯ Ǯ ǯ

D.2.1.7 Step 5: Assemble Test Sets (TD5)

‹…‡ƒŽŽ–‡•–…ƒ•‡•ƒ”‡‹˜ƒŽ‹†‹–Šƒ––Š‡›•Š‘—Ž†…ƒ—•‡ƒˆƒ—Ž–‡••ƒ‰‡–‘ƒ’’‡ƒ”ǡ–Š‡›…ƒƒŽŽ„‡’Žƒ…‡†
into the one test set as follows:
TS1: TEST CASES 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16.

D.2.1.8 Step 6: Derive Test Procedures (TD6)

Ž›‘‡’”‘…‡†—”‡‹•”‡“—‹”‡†ƒ•ˆ‘ŽŽ‘™•ǣ
ͳǣ…‘˜‡”‹‰–Š‡–‡•–…ƒ•‡‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

D.2.1.9 Error Guessing Test Coverage

As stated in 6.4.1, there is no approach for calculating coverage of test coverage items for error guessing.

128 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Annex E
(informative)

Guidelines and Examples for the Application of Interchangeable

Test Design Techniques

E.1 Guidelines and Examples for Interchangeable Test Design Techniques

E.1.1 Overview
Ž–Š‘—‰Š –Š‡ –‡…Š‹“—‡• ’”‡•‡–‡† ‹ –Š‹• •–ƒ†ƒ”† ƒ”‡ …Žƒ••‹ϐ‹‡† ƒ• •–”—…–—”‡Ǧ„ƒ•‡†ǡ •’‡…‹ϐ‹…ƒ–‹‘Ǧ
„ƒ•‡† ‘” ‡š’‡”‹‡…‡Ǧ„ƒ•‡†ǡ ‹ ’”ƒ…–‹…‡ •‘‡ –‡…Š‹“—‡• …ƒ „‡ —•‡† ‹–‡”…Šƒ‰‡ƒ„Ž› ȋ•‡‡ 5.1). This
‹• †‡‘•–”ƒ–‡† ‹ –Š‡ ˆ‘ŽŽ‘™‹‰ ‡šƒ’Ž‡ǡ ™Š‹…Š ‹ŽŽ—•–”ƒ–‡• Š‘™ „”ƒ…Š –‡•–‹‰ ȋ™Š‹…Š ‹• …‘‘Ž›
”‡ˆ‡””‡†–‘ƒ•ƒ•–”—…–—”‡Ǧ„ƒ•‡†–‡…Š‹“—‡Ȍ…ƒ„‡ƒ’’Ž‹‡†ˆ‘”•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†–‡•–‹‰Ǥ

ǤͳǤʹ ”ƒ…Š‡•–‹‰ƒ•ƒ’‡…‹ϐ‹…ƒ–‹‘Ǧƒ•‡†‡…Š‹“—‡

ǤͳǤʹǤͳ ’‡…‹ϐ‹…ƒ–‹‘

‘•‹†‡”–Š‡ˆ‘ŽŽ‘™‹‰‡šƒ’Ž‡•’‡…‹ϐ‹…ƒ–‹‘ǡ™Š‹…Š†‡ϐ‹‡•ƒŽ‘‰‹ˆ—…–‹‘–Šƒ––ƒ‡•ƒ—•‡”ƒ‡ƒ†
password as input to determine whether the user is valid:
The component shall ask for a username and a password. The user must enter a correct username
followed by a corresponding password to be logged into the system. The user is given three attempts
each and 20 seconds each to enter the username and password. If the user does not enter both the
username and the password within 3 tries each and within 20 seconds each, the system will be locked
and disallow any further attempts to login.
Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’Šˆ‘”–Š‹•…‘’‘‡–‹••Š‘™„‡Ž‘™ǣ

[time > 20 seconds OR try = 3] End

(system locked)
B6

[time > 20 seconds

OR try = 3]

End
Start Get username Get password
B3 B5 (logged in)
B1 B2 [username OK] B4 [password OK] B7

[wrong username [wrong password

AND try < 3

‹‰—”‡ǤͳȄ‘–”‘ŽϐŽ‘™‰”ƒ’Šˆ‘”ƒŽ‘‰‹ˆ—…–‹‘

E.1.2.2 Step 1: Identify Feature Sets (TD1)

•–Š‡”‡‹•‘Ž›‘‡–‡•–‹–‡†‡ϐ‹‡†‹–Š‡–‡•–„ƒ•‹•ǡ‘Ž›‘‡ˆ‡ƒ–—”‡•‡–‡‡†•–‘„‡†‡ϐ‹‡†ǣ
FS1: login function

© ISO/IEC 2015 – All rights reserved 129

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

E.1.2.3 Step 2: Derive Test Conditions (TD2)

Š‡ …‘–”‘Ž ϐŽ‘™ ‰”ƒ’Š ’”‘˜‹†‡• ƒ ‰‘‘† ‘˜‡”˜‹‡™ ‘ˆ –Š‡ ˆ—…–‹‘ƒŽ‹–› ƒ† ƒ‹†• –Š‡ ‹†‡–‹ϐ‹…ƒ–‹‘ ‘ˆ
–‡•–…‘†‹–‹‘•ˆ‘”„”ƒ…Š–‡•–‹‰ǤŠ‡•“—ƒ”‡„Ž‘…•‹–Š‡…‘–”‘ŽϐŽ‘™‰”ƒ’Š”‡’”‡•‡–•‡‰‡–•‘ˆ
program source code in the component, while the diamonds represent decisions. Arrows out of each
diamond represent decision outcomes. The possible transfers of control are:

ͳ՜ʹ ͵՜Ͷ ͷ՜Ͷ

ʹ՜͵ ͵՜͸ ͷ՜͸
͵՜ʹ Ͷ՜ͷ ͷ՜͹
‘”„”ƒ…Š…‘˜‡”ƒ‰‡ǡ–Š‡–‡•–…‘†‹–‹‘•ƒ”‡–Š‡„”ƒ…Š‡•ȋƒ”…•Ȍ–Šƒ–ƒ”‡”‡’”‡•‡–‡†„›ƒ””‘™•‹–Š‡
…‘–”‘ŽϐŽ‘™‰”ƒ’ŠǤŠ‡”‡ƒ”‡‹‡‹–‘–ƒŽǡƒ•ˆ‘ŽŽ‘™•ǣ

TCOND1: ͳ՜ʹ (for FS1)

TCOND2: ʹ՜͵ (for FS1)

TCOND3: ͵՜ʹ (for FS1)

TCOND4: ͵՜Ͷ (for FS1)

TCOND5: ͵՜͸ (for FS1)

TCOND6: Ͷ՜ͷ (for FS1)

TCOND7: ͷ՜Ͷ (for FS1)

TCOND8: ͷ՜͸ (for FS1)

TCOND9: ͷ՜͹ (for FS1)

E.1.2.4 Step 3: Derive Test Coverage Items (TD3)

‘” „”ƒ…Š …‘˜‡”ƒ‰‡ǡ –Š‡ –‡•– …‘˜‡”ƒ‰‡ ‹–‡• ƒ”‡ –Š‡ „”ƒ…Š‡• ‹ –Š‡ …‘–”‘Ž ϐŽ‘™ ‰”ƒ’Šǡ ™Š‹…Š
are the same as the test conditions. In this example there are nine test coverage items for branch
coverage, as follows:

TCOVER1: ͳ՜ʹ (for TCOND1)

TCOVER2: ʹ՜͵ (for TCOND2)

TCOVER3: ͵՜ʹ (for TCOND3)

TCOVER4: ͵՜Ͷ (for TCOND4)

TCOVER5: ͵՜͸ (for TCOND5)

TCOVER6: Ͷ՜ͷ (for TCOND6)

TCOVER7: ͷ՜Ͷ (for TCOND7)

TCOVER8: ͷ՜͸ (for TCOND8)

TCOVER9: ͷ՜͹ (for TCOND9)

E.1.2.5 Step 4: Derive Test Cases (TD4)

‡•–…ƒ•‡•ˆ‘”„”ƒ…Š–‡•–‹‰ƒ”‡†‡”‹˜‡†„›‹†‡–‹ˆ›‹‰…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Š•–Šƒ–”‡ƒ…Š‘‡‘”‘”‡
„”ƒ…Š‡•ȋ–‡•–…‘˜‡”ƒ‰‡‹–‡•Ȍ–Šƒ–Šƒ˜‡‘–›‡–„‡‡‡š‡…—–‡††—”‹‰–‡•–‹‰ǡ†‡–‡”‹‹‰‹’—–•–Šƒ–

130 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

exercise those sub-paths, determining the expected result of each test, and repeating until the required
Ž‡˜‡Ž‘ˆ–‡•–…‘˜‡”ƒ‰‡‹•ƒ…Š‹‡˜‡†Ǥ ‘”„”ƒ…Š…‘˜‡”ƒ‰‡ǡƒ›‹†‹˜‹†—ƒŽ–‡•–…ƒ•‡™‹ŽŽ‡š‡”…‹•‡‘‡•—„Ǧ
’ƒ–Šƒ†Š‡…‡’‘–‡–‹ƒŽŽ›ƒ›†‡…‹•‹‘•ƒ†„”ƒ…Š‡•Ǥ
‘•‹†‡”ƒ–‡•–…ƒ•‡–Šƒ–‡š‡…—–‡•–Š‡•—„Ǧ’ƒ–Šͳ՜ʹ՜͵՜Ͷ՜ͷ՜͹ǤŠ‹•…ƒ•‡ƒ”‹•‡•™Š‡ƒ˜ƒŽ‹†
—•‡”ƒ‡ƒ†ƒ–…Š‹‰’ƒ••™‘”†ƒ”‡‡ƒ…Š’”‘˜‹†‡†‘–Š‡ϐ‹”•–ƒ––‡’–ǤŠ‡’ƒ–Š‡š‡…—–‡•ͷ‘—–‘ˆ
the 9 branches, giving 56% coverage (which is not the same as coverage for the decisions).
‘™…‘•‹†‡”ƒ–‡•–…ƒ•‡™Š‹…Š‡š‡…—–‡•–Š‡•—„Ǧ’ƒ–Šͳ՜ʹ՜͵՜ʹ՜͵՜Ͷ՜ͷ՜Ͷ՜ͷ՜͸ǤŠ‹•
sub-path arises when an invalid username and an incorrect password are provided and the correct
password is not provided within the 20 second time limit. Now all branches have been covered except
ˆ‘”͵՜͸ǤŠ‹••—„Ǧ’ƒ–Š…ƒ„‡…‘˜‡”‡†‹ˆǡˆ‘”‡šƒ’Ž‡ǡƒ†‹˜ƒŽ‹†—•‡”ƒ‡‹•’”‘˜‹†‡†–Š”‡‡–‹‡•
in a row or waiting too long entering a valid one. Then all branches have been covered, including all
decisions. Note that some conditions within decisions have not been covered.
‡•–…ƒ•‡•–‘…‘˜‡”‡ƒ…Š…‘–”‘ŽϐŽ‘™•—„Ǧ’ƒ–Šƒ”‡•Š‘™„‡Ž‘™ǣ

Table E.1 — Test cases for login function

Inputs
Test Username Password Test Coverage Expected
Case Username Wait time Password Wait Time Sub-path Items Result
1 †› ζʹͲ Warhol ζʹͲ ͳ՜ʹ՜͵՜Ͷ TCOVER1, Logged in
՜ͷ՜͹ TCOVER2,
TCOVER4,
TCOVER6,
TCOVER9
2 InVaLiD ζʹͲ - - ͳ՜ʹ՜͵՜ʹ TCOVER1, ›•–‡
InVAliD ζʹͲ ՜͵՜Ͷ՜ͷ locked
†› ζʹͲ TCOVER2,
՜Ͷ՜ͷ՜͸
Warhol ηʹͳ
TCOVER3,
TCOVER4,
TCOVER6,
TCOVER7,
TCOVER8
3 ”ƒ†› ηʹͳ - ͳ՜ʹ՜͵՜͸ TCOVER1, ›•–‡
locked
TCOVER2,
TCOVER5

E.1.2.6 Step 5: Assemble Test Sets (TD5)

‹…‡–Š‡”‡ƒ”‡‘Ž›–Š”‡‡–‡•–…ƒ•‡•”‡“—‹”‡†ǡ™‡ƒ›…Š‘‘•‡–‘…‘„‹‡–Š‡‹–‘–Š‡‘‡–‡•–•‡–ǣ
TS1: TEST CASES 1, 2 and 3.

E.1.2.7 Step 6: Derive Test Procedures (TD6)

‹…‡–Š‡”‡ƒ”‡‘Ž›–Š”‡‡–‡•–…ƒ•‡•ƒ†‘‡–‡•–•‡–ǡ™‡ƒ›…Š‘‘•‡–‘†‡ϐ‹‡‘Ž›‘‡–‡•–’”‘…‡†—”‡ǣ
ͳǣ…‘˜‡”‹‰ƒŽŽ–‡•–…ƒ•‡•‹ͳǡ‹–Š‡‘”†‡”•’‡…‹ϐ‹‡†‹–Š‡–‡•–•‡–Ǥ

© ISO/IEC 2015 – All rights reserved 131

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

E.1.2.8 Branch Testing Coverage

Using the formula provided in 6.3.2 and the test coverage items derived above:
9
Coverage( branch) = × 100% = 100%
9
Thus, 100% coverage of test coverage items for branch testing has been achieved.

132 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Annex F
(informative)

Test Design Technique Coverage Effectiveness

F.1 Test Design Technique Coverage Effectiveness

F.1.1 Guidance
Up to this point this standard has provided no guidance on either the choice of test design techniques
‘”–‡•–…‘’Ž‡–‹‘…”‹–‡”‹ƒȋ•‘‡–‹‡•‘™ƒ•–‡•–ƒ†‡“—ƒ…›…”‹–‡”‹ƒȌǡ‘–Š‡”–Šƒ–Šƒ––Š‡›•Š‘—Ž†
be selected from Clauses 5 and 6”‡•’‡…–‹˜‡Ž›ǤŠ‡ƒ‹”‡ƒ•‘ˆ‘”–Š‹•‹•–Šƒ––Š‡”‡‹•‘‡•–ƒ„Ž‹•Š‡†
…‘•‡•—• ‘ ™Š‹…Š –‡…Š‹“—‡• ƒ† …”‹–‡”‹ƒ ƒ”‡ –Š‡ ‘•– ‡ˆˆ‡…–‹˜‡Ǥ Š‡ ‘Ž› …‘•‡•—• ‹• –Šƒ– –Š‡
•‡Ž‡…–‹‘™‹ŽŽ˜ƒ”›ƒ•‹–•Š‘—Ž†„‡†‡’‡†‡–‘ƒ—„‡”‘ˆˆƒ…–‘”••—…Šƒ•”‹•ǡ…”‹–‹…ƒŽ‹–›ǡƒ’’Ž‹…ƒ–‹‘
area, and cost. Research into the relative effectiveness of test case design and measurement techniques
Šƒ•ǡ•‘ˆƒ”ǡ’”‘†—…‡†‘†‡ϐ‹‹–‹˜‡”‡•—Ž–•ƒ†ƒŽ–Š‘—‰Š•‘‡‘ˆ–Š‡–Š‡‘”‡–‹…ƒŽ”‡•—Ž–•ƒ”‡’”‡•‡–‡†
„‡Ž‘™‹–•Š‘—Ž†„‡”‡…‘‰‹•‡†–Šƒ––Š‡›–ƒ‡‘ƒ……‘—–‘ˆ…‘•–Ǥ
There is no requirement to choose corresponding test case design and test coverage measurement
ƒ’’”‘ƒ…Š‡•Ǥ ’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡† –‡•– †‡•‹‰ –‡…Š‹“—‡• ƒ”‡ ‡ˆˆ‡…–‹˜‡ ƒ– †‡–‡…–‹‰ ‡””‘”• ‘ˆ ‘‹••‹‘ǡ
™Š‹Ž‡•–”—…–—”‡Ǧ„ƒ•‡†–‡•–†‡•‹‰–‡…Š‹“—‡•…ƒ‘Ž›†‡–‡…–‡””‘”•‘ˆ…‘‹••‹‘Ǥ‘ƒ–‡•–’Žƒ…‘—Ž†
–›’‹…ƒŽŽ›”‡“—‹”‡„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•–‘„‡—•‡†–‘‰‡‡”ƒ–‡ƒ‹‹–‹ƒŽ•‡–‘ˆ–‡•–…ƒ•‡•ǡ™Š‹Ž‡ƒŽ•‘
”‡“—‹”‹‰ ͳͲͲΨ „”ƒ…Š …‘˜‡”ƒ‰‡ –‘ „‡ ƒ…Š‹‡˜‡†Ǥ Š‹• †‹˜‡”•‡ ƒ’’”‘ƒ…Š …‘—Ž†ǡ ’”‡•—ƒ„Ž›ǡ Ž‡ƒ† –‘
„”ƒ…Š–‡•–‹‰„‡‹‰—•‡†–‘‰‡‡”ƒ–‡ƒ›•—’’Ž‡‡–ƒ”›–‡•–…ƒ•‡•”‡“—‹”‡†–‘ƒ…Š‹‡˜‡…‘˜‡”ƒ‰‡‘ˆƒ›
„”ƒ…Š‡•‹••‡†„›–Š‡„‘—†ƒ”›˜ƒŽ—‡ƒƒŽ›•‹•–‡•–…ƒ•‡•—‹–‡Ǥ
†‡ƒŽŽ›–Š‡–‡•–…‘˜‡”ƒ‰‡Ž‡˜‡Ž•…Š‘•‡ƒ•–‡•–…‘’Ž‡–‹‘…”‹–‡”‹ƒ•Š‘—Ž†ǡ™Š‡”‡˜‡”’‘••‹„Ž‡ǡ„‡ͳͲͲΨǤ
–”‹…– †‡ϐ‹‹–‹‘• ‘ˆ –‡•– …‘˜‡”ƒ‰‡ Ž‡˜‡Ž• Šƒ˜‡ •‘‡–‹‡• ƒ†‡ –Š‹• Ž‡˜‡Ž ‘ˆ …‘˜‡”ƒ‰‡ ‹’”ƒ…–‹…ƒ„Ž‡ǡ
Š‘™‡˜‡”–Š‡†‡ϐ‹‹–‹‘•‹clause 6Šƒ˜‡„‡‡†‡ϐ‹‡†–‘ƒŽŽ‘™‹ˆ‡ƒ•‹„Ž‡…‘˜‡”ƒ‰‡‹–‡•–‘„‡†‹•…‘—–‡†
from the calculations thus making 100% coverage an achievable goal.
With test completion criteria of 100% (and only 100%) it is possible to relate some of them in an
ordering, where criteria are shown to subsume, or include, other criteria. One criterion is said to
•—„•—‡ƒ‘–Š‡”‹ˆǡˆ‘”ƒŽŽ–‡•–‹–‡•ƒ†–Š‡‹”–‡•–„ƒ•‡•ǡ‡˜‡”›–‡•–…ƒ•‡•—‹–‡–Šƒ–•ƒ–‹•ϐ‹‡•–Š‡ϐ‹”•–
…”‹–‡”‹‘ƒŽ•‘•ƒ–‹•ϐ‹‡•–Š‡•‡…‘†Ǥ ‘”‡šƒ’Ž‡ǡ„”ƒ…Š…‘˜‡”ƒ‰‡•—„•—‡••–ƒ–‡‡–…‘˜‡”ƒ‰‡„‡…ƒ—•‡
if branch coverage is achieved (to 100%), then statement coverage to 100% will be achieved as well.
–•Š‘—Ž†„‡‘–‡†–Šƒ––Š‡Dz•—„•—‡•dz”‡Žƒ–‹‘†‡•…”‹„‡†Š‡”‡•–”‹…–Ž›Ž‹•–‡•–…‘˜‡”ƒ‰‡…”‹–‡”‹ƒȋ”ƒ–Š‡”
–Šƒ–‡•–†‡•‹‰–‡…Š‹“—‡•Ȍƒ†•‘‘Ž›’”‘˜‹†‡•ƒ‹†‹”‡…–‹†‹…ƒ–‹‘‘ˆ–Š‡”‡Žƒ–‹˜‡‡ˆˆ‡…–‹˜‡‡••‘ˆ
test design techniques.
‘–ƒŽŽ–‡•–…‘˜‡”ƒ‰‡…”‹–‡”‹ƒ…ƒ„‡”‡Žƒ–‡†„›–Š‡•—„•—‡•‘”†‡”‹‰ƒ†–Š‡•’‡…‹ϐ‹…ƒ–‹‘Ǧ„ƒ•‡†ƒ†
structure-based criteria are not related at all. A partial ordering of criteria is possible for structure-
based test design techniques, as illustrated in Figure F.1 below, where an arrow from one criterion to
ƒ‘–Š‡”‹†‹…ƒ–‡•–Šƒ––Š‡ϐ‹”•–…”‹–‡”‹‘•—„•—‡•–Š‡•‡…‘†ǤŠ‡”‡ƒ–‡•–…‘˜‡”ƒ‰‡…”‹–‡”‹‘†‘‡•‘–
ƒ’’‡ƒ”‹–Š‡’ƒ”–‹ƒŽ‘”†‡”‹‰•–Š‡‹–‹•‘–”‡Žƒ–‡†–‘ƒ›‘–Š‡”…”‹–‡”‹‘„›–Š‡•—„•—‡•”‡Žƒ–‹‘Ǥ

© ISO/IEC 2015 – All rights reserved 133

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Figure F.1 — Partial Ordering of Structural Test Coverage Criteria (Reid 1996)

Despite its intuitive appeal the subsumes relation suffers a number of limitations that should be
considered before using it to choose test completion criteria:
Ȅ ‹”•–Ž›‹–”‡Žƒ–‡•‘Ž›ƒ•—„•‡–‘ˆ–Š‡ƒ˜ƒ‹Žƒ„Ž‡–‡•–…‘’Ž‡–‹‘…”‹–‡”‹ƒƒ†‹…Ž—•‹‘‹–Š‹••—„•‡–
’”‘˜‹†‡•‘‹†‹…ƒ–‹‘‘ˆ‡ˆˆ‡…–‹˜‡‡••ǡ•‘‘–Š‡”…”‹–‡”‹ƒ‘–•Š‘™‹–Š‡ϐ‹‰—”‡ƒ„‘˜‡•Š‘—Ž†•–‹ŽŽ
be considered.
Ȅ ‡…‘†Ž›ǡ–Š‡•—„•—‡•”‡Žƒ–‹‘’”‘˜‹†‡•‘‡ƒ•—”‡‘ˆ–Š‡ƒ‘—–„›™Š‹…Š‘‡…”‹–‡”‹‘•—„•—‡•
ƒ‘–Š‡”ƒ†•—„•‡“—‡–Ž›†‘‡•‘–’”‘˜‹†‡ƒ›‡ƒ•—”‡‘ˆ”‡Žƒ–‹˜‡…‘•–‡ˆˆ‡…–‹˜‡‡••Ǥ
Ȅ Š‹”†Ž›ǡ–Š‡’ƒ”–‹ƒŽ‘”†‡”‹‰•‘Ž›ƒ’’Ž›–‘•‹‰Ž‡…”‹–‡”‹ƒ™Š‹Ž‡‹–‹•”‡…‘‡†‡†–Šƒ–‘”‡–Šƒ
one criterion is used, with at least one functional and one structural criterion.
Ȅ ‹ƒŽŽ›ǡ ƒ† ‘•– ‹’‘”–ƒ–Ž›ǡ –Š‡ •—„•—‡• ”‡Žƒ–‹‘ †‘‡• ‘– ‡…‡••ƒ”‹Ž› ‘”†‡” –‡•– …‘’Ž‡–‹‘
…”‹–‡”‹ƒ‹–‡”•‘ˆ–Š‡‹”ƒ„‹Ž‹–›–‘‡š’‘•‡ˆƒ—Ž–•ȋ–Š‡‹”–‡•–‡ˆˆ‡…–‹˜‡‡••ȌǤ–Šƒ•„‡‡•Š‘™ǡˆ‘”
‹•–ƒ…‡ǡ–Šƒ–ͳͲͲΨ’ƒ–Š…‘˜‡”ƒ‰‡ȋ™Š‡ƒ…Š‹‡˜ƒ„Ž‡Ȍƒ›‘–„‡ƒ•‡ˆˆ‡…–‹˜‡ǡˆ‘”•‘‡–‡•–‹–‡•ǡ
ƒ••‘‡‘ˆ–Š‡…”‹–‡”‹ƒ‹–•—„•—‡•ǡ•—…Šƒ•–Š‘•‡…‘…‡”‡†™‹–Š†ƒ–ƒϐŽ‘™ǤŠ‹•‹•„‡…ƒ—•‡•‘‡
‡””‘”• ƒ”‡ †ƒ–ƒ •‡•‹–‹˜‡ ƒ† ™‹ŽŽ ‘– „‡ ‡š’‘•‡† „› •‹’Ž› ‡š‡…—–‹‰ –Š‡ ’ƒ–Š ‘ ™Š‹…Š –Š‡›
Ž‹‡ǡ „—– ”‡“—‹”‡ ˜ƒ”‹ƒ„Ž‡• –‘ –ƒ‡ ƒ ’ƒ”–‹…—Žƒ” ˜ƒŽ—‡ ƒ• ™‡ŽŽ ȋ‡Ǥ‰Ǥ  Dz—’”‘–‡…–‡†dz †‹˜‹•‹‘ „› ƒ
‹–‡‰‡”˜ƒ”‹ƒ„Ž‡ƒ›‡””‘‡‘—•Ž›„‡‹…Ž—†‡†‹ƒ–‡•–‹–‡–Šƒ–™‹ŽŽ‘Ž›ˆƒ‹Ž‹ˆ–Šƒ–˜ƒ”‹ƒ„Ž‡–ƒ‡•
ƒ‡‰ƒ–‹˜‡˜ƒŽ—‡ȌǤƒ–‹•ˆ›‹‰†ƒ–ƒϐŽ‘™…”‹–‡”‹ƒ…ƒ…‘…‡–”ƒ–‡–Š‡–‡•–‹‰‘–Š‡•‡ƒ•’‡…–•‘ˆƒ–‡•–
‹–‡ǯ•„‡Šƒ˜‹‘—”ǡ–Š—•‹…”‡ƒ•‹‰–Š‡’”‘„ƒ„‹Ž‹–›‘ˆ‡š’‘•‹‰•—…Š‡””‘”•Ǥ–…ƒ„‡•Š‘™–Šƒ–‹
•‘‡…‹”…—•–ƒ…‡•–‡•–‡ˆˆ‡…–‹˜‡‡••‹•‹…”‡ƒ•‡†„›–‡•–‹‰ƒ•—„•‡–‘ˆ–Š‡’ƒ–Š•”‡“—‹”‡†„›ƒ
particular criteria but exercising this subset with more test cases.
Š‡•—„•—‡•”‡Žƒ–‹‘‹•Š‹‰ŠŽ›†‡’‡†‡–‘–Š‡†‡ϐ‹‹–‹‘‘ˆˆ—ŽŽ…‘˜‡”ƒ‰‡ˆ‘”ƒ…”‹–‡”‹‘ƒ†ƒŽ–Š‘—‰Š
–Š‡ ϐ‹‰—”‡ ƒ„‘˜‡ ‹• …‘””‡…– ˆ‘” –Š‡ †‡ϐ‹‹–‹‘• ‹ Clause 5 ‹– ƒ› ‘– ƒ’’Ž› –‘ ƒŽ–‡”ƒ–‹˜‡ †‡ϐ‹‹–‹‘•
used elsewhere.

134 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Annex G
(informative)

ISO/IEC/IEEE 29119-4 and BS 7925-2 Test Design Technique

Alignment

This annex describes the alignment of the test design techniques in this part of ISO/IEC/IEEE 29119
and BS 7925-2.

Table G.1 — BS 7925-2:1998 to ISO/IEC/IEEE 29119-4 test design technique mapping

BS 7925-2:1998 ISO/IEC/IEEE 29119-4

Test Design Techniques
3.1 Equivalence Partitioning 5.2.1 Equivalence Partitioning
3.2 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹• 5.2.3 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹•
3.3 State Transition Testing 5.2.8 State Transition Testing
3.4 Cause-Effect Graphing 5.2.7 Cause-Effect Graphing
3.5 ›–ƒš‡•–‹‰ 5.2.4 ›–ƒš‡•–‹‰
3.6 Statement Testing 5.3.1 Statement Testing
5.3.2 Branch Testing
3.7 Branch/Decision Testing
5.3.3 Decision Testing
3.8 Data Flow Testing 5.3.7 Data Flow Testing
3.9 Branch Condition Testing 5.3.4 Branch Condition Testing
3.10 Branch Condition Combination Testing 5.3.5 Branch Condition Combination Testing
‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡
3.11 ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‡•–‹‰ 5.3.6
ȋȌ‡•–‹‰
3.13 Random Testing 5.2.10 Random Testing
‡•–‡ƒ•—”‡‡–‡…Š‹“—‡•
3.1 Equivalence Partition Coverage 6.2.2 Equivalence Partition Coverage
3.2 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹•‘˜‡”ƒ‰‡ 6.2.3 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹•‘˜‡”ƒ‰‡
3.3 State Transition Coverage 6.2.8 State Transition Testing Coverage
3.4 Cause-Effect Coverage 6.2.7 Cause-Effect Graphing Coverage
3.5 ›–ƒš‘˜‡”ƒ‰‡ 6.2.4 ›–ƒš‡•–‹‰‘˜‡”ƒ‰‡
3.6 Statement Coverage 6.3.1 Statement Testing Coverage
6.3.2 Branch Testing Coverage
3.7 Branch and Decision Coverage
6.3.3 Decision Testing Coverage
3.8 Data Flow Coverage 6.3.7 Data Flow Testing Coverage
3.9 Branch Condition Coverage 6.3.4 Branch Condition Testing Coverage
Branch Condition Combination Testing Cover-
3.10 Branch Condition Combination Coverage 6.3.5
age
‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘ȋȌ‡•–‹‰
3.11 ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‘˜‡”ƒ‰‡ 6.3.6
Coverage
3.13 Random Testing 6.2.10 Random Testing Coverage

© ISO/IEC 2015 – All rights reserved 135

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

Table G.1 (continued)

BS 7925-2:1998 ISO/IEC/IEEE 29119-4

—‹†‡Ž‹‡•ˆ‘”–Š‡’’Ž‹…ƒ–‹‘‘ˆ‡•–‡•‹‰Ƭ‡ƒ•—”‡‡–‡…Š‹“—‡•
B.1 Equivalence Partitioning B.2.1 Equivalence Partitioning
B.2 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹• B.2.3 ‘—†ƒ”›ƒŽ—‡ƒŽ›•‹•
B.3 State Transition Testing B.2.8 State Transition Testing
B.4 Cause-Effect Graphing B.2.7 Cause-Effect Graphing
B.5 ›–ƒš‡•–‹‰ B.2.4 ›–ƒš‡•–‹‰
B.6 Statement Testing C.2.1 Statement Testing
B.7 Branch/Decision Testing C.2.2 Branch/Decision Testing
B.8 Data Flow Testing C.2.4 Data Flow Testing
B.9 Branch Condition Testing
Branch Condition Testing, Branch Condition
B.10 Branch Condition Combination Testing C.2.3 ‘„‹ƒ–‹‘‡•–‹‰ƒ†‘†‹ϐ‹‡†‘†‹–‹‘
‡…‹•‹‘‘˜‡”ƒ‰‡ȋȌ‡•–‹‰
B.11 ‘†‹ϐ‹‡†‘†‹–‹‘‡…‹•‹‘‡•–‹‰
B.13 Random Testing B.2.10 Random Testing
Test Technique Effectiveness
Annex C Test Technique Effectiveness Annex F Test Design Technique Coverage Effectiveness

136 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015(E)

Bibliography

ȏͳȐ Ćęč
ǤǡƬĈĆĞ ǤŠ‡‘ˆ–™ƒ”‡‡•–‰‹‡‡”ǯ•ƒ†„‘‘Ǥǯ‡‹ŽŽ›‡†‹ƒǡ…ǡʹͲͲͺ
ȏʹȐ ĊĎğĊėǤŽƒ…‘š‡•–‹‰Ǥ‡…Š‹“—‡•ˆ‘” —…–‹‘ƒŽ‡•–‹‰‘ˆ‘ˆ–™ƒ”‡ƒ†›•–‡•Ǥ ‘Š
‹Ž‡›Ƭ‘•…ǡͳͻͻͷ
ȏ͵Ȑ Non-Functional TestingǤ ”‹–‹•Š ‘’—–‡” ‘…‹‡–› ’‡…‹ƒŽ –‡”‡•–
”‘—’ ‹ ‘ˆ–™ƒ”‡
‡•–‹‰ǡ ȏ˜‹‡™‡† ͹ ‡’–‡„‡” ʹͲͳͳȐǤ ˜ƒ‹Žƒ„Ž‡ ˆ”‘ http://www.testingstandards.
co.—Ȁ‘̴ˆ—…–‹‘ƒŽ̴–‡•–‹‰̴–‡…Š‹“—‡•.htm
ȏͶȐ ǤͳͻͻͺǤ͹ͻʹͷǦʹǣͳͻͻͺǡSoftware testing – Software component
testing. Available from: http://shop.bsigroup.com/
ȏͷȐ ĚėēĘęĊĎē I. Practical Software Testing: A Process-Oriented Approach. Springer-Verlag, 2003
ȏ͸Ȑ ĔĕĊđĆēĉ L. A Practitioner’s Guide to Software Test Design. Artech House, Inc, 2004
ȏ͹Ȑ čĔǤǤ—ƒŽ‹–›”‘‰”ƒ‹‰Ǥ‹Ž‡›ǡͳͻͺ͹
ȏͺȐ čĔĜ  ǤǤ  ͳͻ͹ͺǤ ‡•–‹‰ ‘ˆ–™ƒ”‡ ‡•‹‰ ‘†‡ŽŽ‡† „› ‹‹–‡Ǧ–ƒ–‡ ƒ…Š‹‡•Ǥ ǣ IEEE
Transactions on Software Engineering, Vol. SE-4(3).
ȏͻȐ ėĆĎČǤǡƬ ĆĘĐĎĊđǤ›•–‡ƒ–‹…‘ˆ–™ƒ”‡‡•–‹‰Ǥ”–‡…Š‘—•‡…ǡʹͲͲʹ
ȏͳͲȐ ĊĘĎĐĆēǤǡƬĆĒĊĘč G. Software Testing: Principles and Practices. Pearson Education, 2007
ȏͳͳȐ
ėĎēĉĆđǤǡċċĚęę J., AēĉđĊėǤʹͲͲͷǤ‘„‹ƒ–‹‘‡•–‹‰–”ƒ–‡‰‹‡•ǣ—”˜‡›ǤǣSoftware
‡•–‹‰ǡ‡”‹ϔ‹…ƒ–‹‘ƒ†‡Ž‹ƒ„‹Ž‹–›ǡ ‘Š‹Ž‡›Ƭ‘•–†Ǥǡ15, pp. 167-199.
ȏͳʹȐ
ǤǤƒ†
ǡ
”‹ǡǤǡͳͻͻ͵ǤŽƒ••‹ϐ‹…ƒ–‹‘”‡‡•ˆ‘”ƒ”–‹–‹‘‡•–‹‰Ǥǣ
‘ˆ–™ƒ”‡‡•–‹‰ǡ‡”‹ϔ‹…ƒ–‹‘Ƭ‡Ž‹ƒ„‹Ž‹–›ǡ‹Ž‡›ǡ3(2), pp. 63 - 82.
ȏͳ͵Ȑ ĔēĆĘĘĊēĆĘĘǤǤ
—‹†‡–‘†˜ƒ…‡†‘ˆ–™ƒ”‡‡•–‹‰Ǥ”–‡…Š‘—•‡ǡʹͲͲͺ
ȏͳͶȐ  
  Ȁ
ǤʹͲͲͷǤȀͳͻ͹ͷͻǡ‘ˆ–™ƒ”‡‰‹‡‡”‹‰ǦǦ
—‹†‡–‘
–Š‡‘ˆ–™ƒ”‡‰‹‡‡”‹‰‘†›‘ˆ‘™Ž‡†‰‡ȋȌǤ˜ƒ‹Žƒ„Ž‡ˆ”‘ǡhttp://www.iso.org/
ȏͳͷȐ  
  Ȁ
 Ǥ  ʹͲͳͳǤ Ȁ ʹͷͲͳͲǡ Systems and software engineering
Ȃ ›•–‡ ƒ† •‘ˆ–™ƒ”‡ ’”‘†—…– —ƒŽ‹–› ‡“—‹”‡‡–• ƒ† ˜ƒŽ—ƒ–‹‘ ȋ—ƒȌ Ȃ ›•–‡ ƒ†
software quality models. Available from: http://www.iso.org
ȏͳ͸Ȑ  
  Ȁ
 Ǥ  ʹͲͲ͹Ǥ Ȁ ʹͷͲ͵Ͳǡ oftware engineering – Software
’”‘†—…– —ƒŽ‹–› ‡“—‹”‡‡–• ƒ† ˜ƒŽ—ƒ–‹‘ ȋ—ƒȌ Ȃ —ƒŽ‹–› ”‡“—‹”‡‡–•. Available from:
http://www.iso.org
ȏͳ͹Ȑ ĆēĊė C. Testing Computer Software. TAB Books Inc, 1998
ȏͳͺȐ ĊėēĎČčĆē  ǤǤǡ Ƭ ĎĈčĎĊ  ǤǤ Š‡  ”‘‰”ƒ‹‰ ƒ‰—ƒ‰‡Ǥ  ”‡–‹…‡ǦƒŽŽ ‘ˆ–™ƒ”‡
Series, 1998
ȏͳͻȐ ĎęǤ‘ˆ–™ƒ”‡‡•–‹‰‹–Š‡‡ƒŽ‘”Ž†ǣ’”‘˜‹‰–Š‡”‘…‡••Ǥ”‡••ǡͳͻͻͷ
ȏʹͲȐ Ćēĉđ R. Orthogonal Latin Squares: An Application of Experiment Design to Compiler Testing.
‘—ǤǤͳͻͺͷǡ28 (10) pp. 1054–1058
ȏʹͳȐ ĞĊėĘ
ǤŠ‡”–‘ˆ‘ˆ–™ƒ”‡‡•–‹‰Ǥ ‘Š‹Ž‡›Ƭ‘•…ǡͳͻ͹ͻ

© ISO/IEC 2015 – All rights reserved 137

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ISO/IEC/IEEE 29119-4:2015(E)

ȏʹʹȐ ĚėĘĎĒĚđĚ  Ǥǡ Ƭ ėĔćĊėę  ǤǤ  ͳͻͻͷǤ ƒ—•‡Ǧˆˆ‡…–

”ƒ’Š‹‰ ƒŽ›•‹• ƒ† ƒŽ‹†ƒ–‹‘ ‘ˆ
Requirements. In Proceedings of CASCON’1995.
ȏʹ͵Ȑ ȀǦͳ͹ͺǡ ‘ˆ–™ƒ”‡ ‘•‹†‡”ƒ–‹‘• ‹ ‹”„‘”‡ ›•–‡• ƒ† “—‹’‡– ‡”–‹ϐ‹…ƒ–‹‘Ǥ
RTCA, Inc. 2011.
ȏʹͶȐ ĊĎĉǤͳͻͻ͸Ǥ‘’—Žƒ”‹•…‘…‡’–‹‘•‹‘†—Ž‡‡•–‹‰ǤProceedings of the Software Testing
Conference (STC), Washington DC.

138 © ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
 ISO/IEC/IEEE 29119-4:2015

Important Notices and Disclaimers Concerning IEEE Standards Documents

Notice and Disclaimer of Liability Concerning the Use of IEEE Standards Documents
IEEE Standards documents (standards, recommended practices, and guides), both full-use and trial-use, are developed within IEEE Societies and the Standards Coordinating
Committees of the IEEE Standards Association (“IEEE-SA”) Standards Board. IEEE (“the Institute”) develops its standards through a consensus development process,
approved by the American National Standards Institute (“ANSI”), which brings together volunteers representing varied viewpoints and interests to achieve the final product.
Volunteers are not necessarily members of the Institute and participate without compensation from IEEE. While IEEE administers the process and establishes rules to
promote fairness in the consensus development process, IEEE does not independently evaluate, test, or verify the accuracy of any of the information or the soundness of any
judgments contained in its standards.
IEEE does not warrant or represent the accuracy or content of the material contained in its standards, and expressly disclaims all warranties (express, implied and statutory)
not included in this or any other document relating to the standard, including, but not limited to, the warranties of: merchantability; fitness for a particular purpose; non-
infringement; and quality, accuracy, effectiveness, currency, or completeness of material. In addition, IEEE disclaims any and all conditions relating to: results; and
workmanlike effort. IEEE standards documents are supplied “AS IS” and “WITH ALL FAULTS.”
Use of an IEEE standard is wholly voluntary. The existence of an IEEE standard does not imply that there are no other ways to produce, test, measure, purchase, market, or
provide other goods and services related to the scope of the IEEE standard. Furthermore, the viewpoint expressed at the time a standard is approved and issued is subject to
change brought about through developments in the state of the art and comments received from users of the standard.
In publishing and making its standards available, IEEE is not suggesting or rendering professional or other services for, or on behalf of, any person or entity nor is IEEE
undertaking to perform any duty owed by any other person or entity to another. Any person utilizing any IEEE Standards document, should rely upon his or her own
independent judgment in the exercise of reasonable care in any given circumstances or, as appropriate, seek the advice of a competent professional in determining the
appropriateness of a given IEEE standard.
IN NO EVENT SHALL IEEE BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO: PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE PUBLICATION, USE OF, OR RELIANCE UPON ANY STANDARD, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE AND REGARDLESS OF WHETHER SUCH DAMAGE WAS FORESEEABLE.
Translations
The IEEE consensus development process involves the review of documents in English only. In the event that an IEEE standard is translated, only the English version
published by IEEE should be considered the approved IEEE standard.
Official statements
A statement, written or oral, that is not processed in accordance with the IEEE-SA Standards Board Operations Manual shall not be considered or inferred to be the official
position of IEEE or any of its committees and shall not be considered to be, or be relied upon as, a formal position of IEEE. At lectures, symposia, seminars, or educational
courses, an individual presenting information on IEEE standards shall make it clear that his or her views should be considered the personal views of that individual rather
than the formal position of IEEE.
Comments on standards
Comments for revision of IEEE Standards documents are welcome from any interested party, regardless of membership affiliation with IEEE. However, IEEE does not
provide consulting information or advice pertaining to IEEE Standards documents. Suggestions for changes in documents should be in the form of a proposed change of text,
together with appropriate supporting comments. Since IEEE standards represent a consensus of concerned interests, it is important that any responses to comments and
questions also receive the concurrence of a balance of interests. For this reason, IEEE and the members of its societies and Standards Coordinating Committees are not able
to provide an instant response to comments or questions except in those cases where the matter has previously been addressed. For the same reason, IEEE does not respond to
interpretation requests. Any person who would like to participate in revisions to an IEEE standard is welcome to join the relevant IEEE working group.
Comments on standards should be submitted to the following address:
Secretary, IEEE-SA Standards Board
445 Hoes Lane
Piscataway, NJ 08854 USA
Photocopies
Subject to payment of the appropriate fee, IEEE will grant users a limited, non-exclusive license to photocopy portions of any individual standard for company or
organizational internal use or individual, non-commercial use only. To arrange for payment of licensing fees, please contact Copyright Clearance Center, Customer Service,
222 Rosewood Drive, Danvers, MA 01923 USA; +1 978 750 8400. Permission to photocopy portions of any individual standard for educational classroom use can also be
obtained through the Copyright Clearance Center.
Patents
Attention is called to the possibility that implementation of this standard may require use of subject matter covered by patent rights. By publication of this standard, no
position is taken by the IEEE with respect to the existence or validity of any patent rights in connection therewith. If a patent holder or patent applicant has filed a statement
of assurance via an Accepted Letter of Assurance, then the statement is listed on the IEEE-SA Website at http://standards.ieee.org/about/sasb/patcom/patents.html. Letters of
Assurance may indicate whether the Submitter is willing or unwilling to grant licenses under patent rights without compensation or under reasonable rates, with reasonable
terms and conditions that are demonstrably free of any unfair discrimination to applicants desiring to obtain such licenses.
Essential Patent Claims may exist for which a Letter of Assurance has not been received. The IEEE is not responsible for identifying Essential Patent Claims for which a
license may be required, for conducting inquiries into the legal validity or scope of Patents Claims, or determining whether any licensing terms or conditions provided in
connection with submission of a Letter of Assurance, if any, or in any licensing agreements are reasonable or non-discriminatory. Users of this standard are expressly advised
that determination of the validity of any patent rights, and the risk of infringement of such rights, is entirely their own responsibility. Further information may be obtained
from the IEEE Standards Association.
Participants: The list of IEEE participants can be accessed at the following URL: http://standards.ieee.org/downloads/29119/29119-4-2015/29119-4-2015_wg-
participants.pdf.
IMPORTANT NOTICE: IEEE Standards documents are not intended to ensure safety, health, or environmental protection, or ensure against interference with or from
other devices or networks. Implementers of IEEE Standards documents are responsible for determining and complying with all appropriate safety, security,
environmental, health, and interference protection practices and all applicable laws and regulations.
This IEEE document is made available for use subject to important notices and legal disclaimers. These notices and disclaimers appear in all
publications containing this document and may be found under the heading “Important Notice” or “Important Notices and Disclaimers Concerning
IEEE Documents.” They can also be obtained on request from IEEE or viewed at http://standards.ieee.org/IPR/disclaimers.html.

© ISO/IEC 2015 – All rights reserved

© IEEE 2015 – All rights reserved ͳ͵ͻ

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.
ȀȀʹͻͳͳͻǦͶǣʹͲͳͷȋȌ

Abstract:

5IJT
TUBOEBSE
TVQQPSUT
UFTU
DBTF
EFTJHO
BOE
FYFDVUJPO
EVSJOH
BOZ
QIBTF
PS

UZQF
PG
UFTUJOH
FH
VOJU
JOUFHSBUJPO
TZTUFN
BDDFQUBODF
QFSGPSNBODF

VTBCJMJUZ
SFMJBCJMJUZ


Keywords: 29119-4, life cycle, life cycle process, software, testing

ICS 35.080.00
ISBN 978-0-7381-9841-8 IEEE ISBN: 978-0-7381-9840-8
Price based on 13 pages STD20319
© ISO/IEC 2015 – All rights reserved
© IEEE 2015 – All rights reserved

Authorized licensed use limited to: City College of New York. Downloaded on April 06,2017 at 21:06:51 UTC from IEEE Xplore. Restrictions apply.