SE 433/333 Software Testing & Quality Assurance

SE 433/333 Software Testing
& Quality Assurance
Dennis Mumaugh, Instructor

dmumaugh@depaul.edu
Office: CDM, Room 428
Office Hours: Tuesday, 4:00 – 5:30
April 18, 2017 SE 433: Lecture 4 1 of 101

Administrivia
 Comments and feedback
 Announcements
Solution to Assignment 1 and 2 have been posted to D2L
 Hints
 Look at the Java documentation. For example:
http://docs.oracle.com/javase/8/docs/api/java/lang/Double.html
 Look at the examples (JUnit2.zip) mentioned on the reading list and
provided in D2L.
 In solving a problem, try getting the example working first.

SE 433 – Class 4
Topic:
 Black Box Testing, JUnit Part 2
Reading:
 Pezze and Young: Chapters 9-10
 JUnit documentation: http://junit.org
 An example of parameterized test: JUnit2.zip in D2L
 See also reading list

Assignments 4 and 5
Assignment 4: Parameterized Test
 The objective of this assignment is to develop a parameterized JUnit
test and run JUnit tests using Eclipse IDE. [See JUnit2.zip]
 Due Date: April 25, 2017, 11:59pm
Assignment 5: Black Box Testing – Part 1: Test Case Design
 The objective of this assignment is to design test suites using black-
box techniques to adequately test the programs specified below. You
may use any combination of black-box testing techniques to design
the test cases.
 Due date: May 2, 2017, 11:59pm

Thought for the Day
“More than the act of testing, the act of designing tests is one of the
best bug preventers known. The thinking that must be done to create a
useful test can discover and eliminate bugs before they are coded –
indeed, test-design thinking can discover and eliminate bugs at every
stage in the creation of software, from conception to specification, to
design, coding and the rest.” – Boris Beizer

Assignment 1 Lessons Learned
 The assignment requirements may have some unwritten
items:
 Being able to test the program
 Programs need to “defend” themselves from
 Illegal input
 Boundary conditions

Assignment 1 Part I, Grading rubric
Part I: 10 Points
 Compiles and runs and handles standard tests
 Handles invalid cases -1
 Not a triangle (a + b < c)
 Degenerate triangle (a + b = c), aka line
 Handles illegal input -1

 Negative numbers
 Floating point numbers
 Extra large integers and MAXINT [See above triangle test].
 Text
 If the code handles most of the special condition give it the point.

Assignment 1 Part I, Grading rubric
 For triangles:
 10 + 20 = 30 hence it is a line
 12 + 8 < 30 hence it is not a triangle
 Negative length triangles are bad.
 Also triangles with long numbers such as 2147483647. This number
is 2^31 - 1 ) i.e. MAX_VALUE “A constant holding the maximum
value an int can have, 231-1”.
» The problem is in testing for illegal triangles one must check a + b
<= c and we get arithmetic overflows. [See example on next
slide.]
» I got bit the first time myself.
 The problem in using 21474836471 is simply that it is too large

Assignment 1 Lessons Learned
 Consider:
public class Triangle {
public static void main(String[] args) {
int a = 2147483647;
int b = 2147483647;
int c = 2147483647;
if (a + b <= c) // Check a c b
{
System.out.println("Not a triangle");
} else { System.out.println("Triangle");}
}
}
Result: Not a triangle

Case Study – Knight Capital
High Frequency Trading (HFT)

Case Study – Knight Capital: High Frequency Trading (HFT)
 The Knight Capital Group is an American global financial

services firm.
 Its high-frequency trading algorithms Knight was the largest
trader in U.S. equities
 with a market share of 17.3% on NYSE and 16.9% on NASDAQ.

Case Study – Knight Capital
 Aug 1, 2012.
 In the first 45 minutes, Knight Capital's computers executed a series
of unusually large automatic orders.
“… spit out duplicate buy and sell orders, jamming the market with
high volumes of trades that caused the wild swings in stock prices.”
 By the end of day: $460 million loss
 “Trading Program Ran Amok, With No ‘Off’ Switch”
 In two days, the company's market value plunged by 75%

Case Study – Knight Capital: What Happened?
"Zombie Software" Blamed for Knight Capital Trading Snafu
 A new algorithmic trading program had just been
installed, and began operation on Aug 1.
 A dormant legacy program was somehow "inadvertently
reactivated"
 Once activated, the dormant system started multiplying
stock trades by one thousand
 Sent 4 million orders when attempting to fill just 212
customer orders
 “Knight’s staff looked through eight sets of software
before determining what happened.”

Case Study – Knight Capital: The Investigation and Findings
 SEC launched an investigation in Nov 2012.

Findings:
 Code changes in 2005 introduced defects. Although
the defective function was not meant to be used, it
was kept in.
 New code deployed in late July 2012. The defective
function was triggered under new rules. Unable to
recognize when orders have been filled.
 Ignored system generated warning emails.
 Inadequate controls and procedures for code
deployment and testing.
 Charges filed in Oct 2013
 Knights Capital settled charges for $12 million
Regression Test

Software Evolution
 Change happens throughout the software
development life cycle.
 Before and after delivery
 Change can happen to every aspect of
software
 Changes can affect unchanged areas
» break code, introduce new bugs
» uncover previous unknown bugs
» reintroduce old bugs

Regression Test
 Testing of a previously tested program
following modification to ensure that new
defects have not been introduced or
uncovered in unchanged areas of the
software, as a result of the changes made.
 It should be performed whenever the
software or its environment is changed.
 It applies to testing at all levels.

Regression Test
 Keep a test suite
 Use the test suite after every change
 Compare output with previous tests
 Understand all changes
 If new tests are needed, add to the test suite.

Test Driven Development
(TDD)

Test Early
 Testing should start as early as possible
 design test cases
 Test early has several advantages
 independence from design & code
 discover inconsistencies and
incompleteness of the specifications
 serve as a compendium of the
specifications

Test Driven Development
 Test driven development (TDD) is one of the corner stones
of agile software development
 Agile, iterative, incremental development
 Small iterations, a few units
 Verification and validation carried out for each iteration.
 Design & implement test cases before implementing the functionality
 Run automated regression test of whole system continuously

Process of Test Driven Development
 Tests should be written first (before any code)
 Execute all test cases => all fail
 Implement some functions
 Execute all test cases => some pass
 Repeat implement and re-execute all test cases
 Until all test cases => pass
 Refactoring, to improve design & implementation
 re-execute all test cases => all pass
 Every time changes are made
 re-execute all test cases => all pass

Black Box Testing

Black Box View
 The system is viewed as a black box
 Providesome input
 Observe the output

Functional Testing: A.k.a.: Black Box Testing
 Derive test cases from the functional specifications
functional refers to the source of information

 not to what is tested
 Also known as:

 specification-based testing (from specifications)
 black-box testing (no view of the code)
 Functional specification = description of intended

program behavior
 either formal or informal

Systematic vs. Random Testing
 Random (uniform) testing
 Pick possible inputs randomly and uniformly
 Avoids designer bias
 But treats all inputs as equally valuable
 Systematic (non-uniform) testing
 Select inputs that are especially valuable
 Choose representatives
 Black box testing is systematic testing

Why Not Random Testing?
 Non-uniform distribution of defects
 Example:
 Program: solve a quadratic equation: a x2 + b x + c = 0
 Defect: incomplete implementation logic

does not properly handle special cases: b2 - 4ac = 0 and
a=0
 Failing values are sparse in the input space —
needles in a very big haystack.
 Random sampling is unlikely to choose a=0.0 and b=0.0

Systematic Partition of Input Space
Failures are sparse
Failure ... but dense in some
in the space of
No failure parts of the space
possible inputs ...
The space of possible input values
If we systematically test some Functional testing is one way of

cases from each part, we will drawing lines to isolate regions
include the dense parts with likely failures
The Partition Principle
 Exploit knowledge in problem domain to choose samples for
testing
 Focus on “special” or trouble-prone regions of the input
space
 Failures are sparse in the whole input space ...
 ... but we may find regions in which they are dense
 (Quasi*-) Partition testing
 Separates the input space into classes whose union is
the entire space
 *Quasi because the classes may overlap

The Partition Principle
 Desirable case for partitioning
 Input values that lead to failures are dense (easy
to find) in some classes of input space
 Sampling each class in the quasi-partition by
selecting at least one input value that leads to a
failure, revealing the defect
 Seldom guaranteed, depend on experience-
based heuristics

Black Box Testing
Exploiting the functional specification
 Uses the specification to partition the input
space
 e.g., specification of “roots” program suggests division
between cases with zero, one, and two real roots
 Test each partition, and boundaries between
partitions
 No guarantees, but experience suggests failures often lie
at the boundaries (as in the “roots” program)

Why Black Box Testing?
 Early.
 can start before code is written
 Effective.
 find some classes of defects, e.g., missing logic
 Widely applicable
 any description of program behavior as spec
 at any level of granularity, from module to system testing.
 Economical
 less expensive than structural (white box) testing
The base-line technique for designing test cases

Early Black Box Testing
 Program code is not necessary
 Only a description of intended behavior is needed
 Even incomplete and informal specifications can be used
» Although precise, complete specifications lead to
better test suites
 Early test design has side benefits
 Often reveals ambiguities and inconsistency in spec
 Useful for assessing testability
» And improving test schedule and budget by improving
spec
 Useful explanation of specification
» or in the extreme case (as in XP), test cases are the
spec

Functional versus Structural: Classes of faults
 Different testing strategies (functional, structural, fault-
based, model-based) are most effective for different classes
of faults
 Functional testing is best for missing logic faults
 A common problem: Some program logic was simply
forgotten
 Structural (code-based) testing will never focus on code
that isn’t there!

Functional vs. Structural Test
 Functional test is applicable in testing at all
granularity levels:
 Unit test (from module interface spec)
 Integration test (from API or subsystem spec)
 System test (from system requirements spec)
 Regression test (from system requirements + bug
history)
 Structural test is applicable in testing relatively
small parts of a system:
 Unit test

Steps: From specification to test cases
1. Decompose the specification
 If the specification is large, break it into independently testable
features to be considered in testing
2. Select representatives
 Representative values of each input, or
 Representative behaviors of a model
• Often simple input/output transformations don’t describe a
system. We use models in program specification, in program
design, and in test design
3. Form test specifications
• Typically: combinations of input values, or model behaviors
4. Produce and execute actual tests

From specification to test cases
Functional
Specifications
Independently
Testable
Feature
Representative
Model
Values
Test Test
Case Cases
Specifications

An Example: Postal Code Lookup
 Input: ZIP code (5-digit US Postal

code)
 Output: List of cities
What are some representative values to
test?

Example: Representative Values
Simple example with

one input, one output
 Correct zip code

 With 0, 1, or many cities
 Malformed zip code
 Empty; 1-4 characters; 6 Note prevalence of boundary
characters; very long values (0 cities, 6 characters)
 Non-digit characters and error cases
 Non-character data

Summary
 Functional testing, i.e., generation of test cases from
specifications is a valuable and flexible approach to
software testing
 Applicable from very early system specs right through module
specifications
 (quasi-)Partition testing suggests dividing the input space
into (quasi-)equivalent classes
 Systematic testing is intentionally non-uniform to address special
cases, error conditions, and other small places
 Dividing a big haystack into small, hopefully uniform piles where the
needles might be concentrated

Basic Techniques of
Black Box Testing

Single Defect Assumption
Failures are rarely the result of the

simultaneous effects of two (or more)
defects.

Functional Testing Concepts
The four key concepts in functional testing are:
 Precisely identify the domain of each input and each output
variable
 Select values from the data domain of each variable having
important properties
 Consider combinations of special values from different input
domains to design test cases
 Consider input values such that the program under test
produces special values from the domains of the output
variables

Developing Test Cases
 Consider: Test cases for input box accepting numbers
between 1 and 1000
 If you are testing for an input box accepting numbers from 1 to 1000
then there is no use in writing thousand test cases for all 1000 valid
input numbers plus other test cases for invalid data.
 Using equivalence partitioning method, above test cases
can be divided into three sets of input data called as
classes. Each test case is a representative of respective
class.
 We can divide our test cases into three equivalence classes
of some valid and invalid inputs.

Developing Test Cases
1. One input data class with all valid inputs. Pick a single
value from range 1 to 1000 as a valid test case. If you
select other values between 1 and 1000 then result is going
to be same. So one test case for valid input data should be
sufficient.
2. Input data class with all values below lower limit. I.e. any
value below 1, as a invalid input data test case.
3. Input data with any value greater than 1000 to represent
third invalid input class.
So using equivalence partitioning you have categorized all
possible test cases into three classes. Test cases with other
values from any class should give you the same result.

Equivalence Classes
 Equivalence classes are the sets of values in a (quasi-)
partition of the input, or output domain
 Values in an equivalence class cause the program to
behave in a similar way:
 failure or success
 Motivation:
 gain a sense of complete testing and avoid redundancy
 First determine the boundaries … then determine the
equivalencies

Determining Equivalence Classes
 Look for ranges of numbers or values
 Look for memberships in groups
 Some may be based on time
 Include invalid inputs
 Look for internal boundaries
 Don’t worry if they overlap with each other —
 better to be redundant than to miss something
 However, test cases will easily overlap with
boundary value test cases

Selecting Data Points
 Determining equivalence classes for each
input variable or field
 Single input variable
 Normal test
» Select one data point from each valid equivalence
class
 Robustness test
» Include invalid equivalence class

Selecting Data Points
 Multiple input variables
 Weak normal test:
class
 Strong normal test:
» Select one data point from each combination of (the
cross product of) the valid equivalence classes
 Weak/strong robustness test:
» Include invalid equivalence classes
 How many test cases do we need?

Example of Selecting Data Points
 Suppose a program has 2 input variables, x and y
 Suppose x can lie in 3 valid equivalence classes:
 a ≤ x < b
 b ≤ x < c
 c ≤ x ≤ d
 Suppose y can lie in 2 valid equivalence classes:
 e ≤ y < f
 f ≤ y ≤ g

Weak Normal Test
 Every normal, i.e., valid, equivalence class of every input
variable is tested in at least one test case.
 A representative value of each normal equivalence class of
each input variable appears in at least one test case.
 Economical, requires few test cases if the values are
selected prudently.
 Complete.

Weak Normal Test
y
a b c d x

Strong Normal Test
 Every combination of normal equivalence classes of every
input variable is tested in at least one test cases.
 More comprehensive.
 Requires more test cases.
 May not be practical for programs with large number of input
variables.

Strong Normal Test
y
a b c d x

Weak Robustness Test
 Add robustness test cases to weak normal test suite.
 Every invalid equivalence class of every input variable is
tested in at least one robustness test case.
 Each robustness test case include only one invalid input
value.
 No combination of invalid input values.

Weak Robustness Test
y
a b c d x

Strong Robustness Test
 Add robustness test cases to strong normal test suite.
 Every invalid equivalence class of an input variable is tested
with all combinations of valid equivalence classes of other
input variable.
 Each robustness test case include only one invalid input
value.
 No combination of invalid input values.

Strong Robustness Test Cases
y
a b c d x

Summary
 For Multiple input variables
 Weak normal test:
class
 Strong normal test:
» Select one data point from each combination of (the
cross product of) the valid equivalence classes
 Weak/strong robustness test:
» Include invalid equivalence classes

Example: nextDate() Function
 This program reads a date in the format of
mm/dd/yyyy
and prints out the next date.
 For example, an input of
03/31/2014
gives an output of
04/01/2014
 A constraint (arbitrary, for illustration purpose only)
 The year is between 1800 and 2200 inclusive

Example: nextDate(): Valid Equivalence Classes
 The valid equivalence classes for the Day
{ 1 ≤ Day ≤ 28 }

 { Day = 29 }
 { Day = 30 }
 { Day = 31 }
 The valid equivalence classes for the Month
 { Month has 30 days }
 { Month has 31 days }
 { Month = February }
 The valid equivalence classes for the Year
 { Year is not a leap year }
 { Year is a leap year }

Example: nextDate(): Invalid Equivalence Classes
 The invalid equivalence classes for the Day
{ Day < 1 } { Day > 31 }
{ Incorrect format of Day } { Illegal characters of Day }
 The invalid equivalence classes for the Month
{ Month < 1 } { Month > 12 }
{ Incorrect format of Month } { Illegal characters of Month }
 The invalid equivalence classes for the Year
{ Year < 1800 } { Year > 2200 }
{ Incorrect format of Year } { Illegal characters of Year }
 Other invalid equivalence classes
{ Incorrect order of Day, Month, Year }
{ Missing Day, Month, or Year }
{ Extra number or character }

Example: nextDate(): Test Cases: Weak Normal
 Valid equivalence classes and
data points
 Day Data Points
» { 1 ≤ Day ≤ 28 } 10
» { Day = 29 } 29
 Weak normal
» { Day = 30 } 30 test cases (4
» { Day = 31 } 31 cases)
 Month
» { Month has 30 days } 04
» { Month has 31 days } 03
1. 02/10/2009
» { Month = February } 02 2. 04/29/2009
 Year 3. 03/30/2008
» { Year is not a leap year } 2009
» { Year is a leap year } 2008
4. 03/31/2008

Example: nextDate(): Test Cases: Strong Normal
 Strong normal test cases (17 cases)
02/10/2008 02/29/2008
02/10/2009
03/10/2008 03/29/2008 03/30/2008 03/31/2008

03/10/2009 03/29/2009 03/30/2009 03/31/2009
04/10/2008 04/29/2008 04/30/2008

04/10/2009 04/29/2009 04/30/2009
 Note: some combinations are invalid, thus excluded
 e.g., 02/30/2008

Example: nextDate(): Test Cases: Weak Robustness
 Add a test case for each invalid equivalence
class
 { Day < 1 } 02/00/2008
 { Day > 31 } 03/36/2009
 { Incorrect format of Day } 02/7/2008
 { Illegal characters of Day } 02/First/2008
 { Month < 1 } 00/10/2009
 { Month > 12 } 15/10/2008
 { Incorrect format of Month } 3/10/2008
 { Illegal characters of Month } Mar/10/2009
 { Year < 1800 } 02/10/1745
 { Year > 2200 } 02/10/2350
 { Incorrect format of Year } 02/10/10
 { Illegal characters of Year } 02/10/’00
 { Incorrect order of Day, Month, Year } 29/03/2008
 { Missing Day, Month, or Year } 02/10
 { Extra number or character } 02/20/2008/2009

Example: nextDate(): Test Cases: Strong Robustness
 Add invalid test cases resulting from combination of valid
equivalence classes
04/31/2008
02/29/2009 02/30/2009 02/31/2009
02/30/2008 02/31/2008
 Ensure each invalid test case contains only one invalid
value.
 Single defect assumption

Boundary Value Testing
 Test values, sizes, or quantities near the design
limits
 value limits, length limits, volume limits
 null strings vs. empty strings
 Errors tend to occur near the extreme values of
inputs (off by one is an example)
 Robustness:
 How does the software react when boundaries are
exceeded?
 Use input values
 at their minimum, just above the minimum
 at a nominal value,
 at the maximum, just below the maximum
Input Boundary Values
 Test cases for a variable x, where a ≤ x ≤ b
a b
x
x(min) x(min+) x(nom) x(max -) x(max)
 Experience shows that errors occur more frequently for

extreme values of a variable.

Input Boundary Values – 2 Variables
Test cases for a variables x1 and x2, where a ≤ x1 ≤ b and c ≤
x2 ≤ d
x2
x1
a b
 Single defect assumption
Example: nextDate() – Test Cases: Boundary Values
 Additional test cases, valid input

04/01/2009 04/30/2009
03/01/2009 03/31/2009
02/01/2009 02/28/2009
02/29/2008
01/01/2008 12/31/2008
01/01/1800 12/31/2200

Robustness Testing
 Test cases for a variable x, where a ≤ x ≤ b
a b
x
 Stress input boundaries

 Acceptable response for invalid inputs?
 Leads to exploratory testing (test hackers)
 Can discover hidden functionality

Robustness Testing – 2 Variables
x2
x1
a b
Example: nextDate() – Test Cases: Boundary Values
 Additional robustness test cases, invalid

input
04/00/2009 04/31/2009
03/00/2009 03/32/2009
02/00/2009 02/29/2009
02/30/2008
01/00/2008 12/32/2008
12/31/1799 01/01/2201
00/01/2009 13/01/2009

Worst-Case Testing
 Discard the single-defect assumption
 Worst-case boundary testing:
 Allow the input values to simultaneously approach their boundaries
 Worst-case robustness testing:
 Allow the input values to simultaneously approach and exceed their
boundaries

Worst Case Boundary Testing – 2 Variables
x2
x1
a b

Worst Case Robustness Testing – 2 Variables
x2
x1
a b

Limitations of Boundary Value Testing
 Doesn’t require much thought
 May miss internal boundaries
 Usually assumes the variables are independent
 Values at the boundary may not have meaning

Special Value Testing
 The most widely practiced form of functional testing
 The tester uses his or her domain knowledge, experience,
or intuition to probe areas of probable errors
 Other terms: “hacking”, “out-of-box testing”, “ad hoc testing”,
“seat of the pants testing”, “guerilla testing”

Uses of Special Value Testing
 Complex mathematical (or algorithmic) calculations
 Worst case situations (similar to robustness)
 Problematic situations from past experience
 “Second guess” the likely implementation

Characteristics of Special Value Testing
 Experience really helps
 Frequently done by the customer or user
 Defies measurement
 Highly intuitive
 Seldom repeatable
 Often, very effective

Summary: Key Concepts
 Black-box testing
 vs. random testing, white-box testing
 Partitioning principle
 Black box testing techniques
 Equivalence class
 Boundary value testing
 Special value testing
 Single defect assumption
 Normal vs. robustness testing
 Weak and strong combinations

Guidelines and observations
 Equivalence Class Testing is appropriate when input data is
defined in terms of intervals and sets of discrete values.
 Equivalence Class Testing is strengthened when combined
with Boundary Value Testing
 Strong equivalence takes the presumption that variables are
independent. If that is not the case, redundant test cases
may be generated

An Introduction to JUnit
Part 2

JUnit Best Practices
 Each test case should be independent.
 Test cases should be independent of
execution order.
 No dependencies on the state of previous
tests.

JUnit Test Fixtures
 The context in which a test case is executed.
 Typically include:
 Common objects or resources that are available
for use by any test case.
 Activities to manage these objects
 Set-up: object and resource allocation
 Tear-down: object and resource de-allocation

Set-Up
 Tasks that must be done prior to each test
case
 Examples:
 Createsome objects to work with
 Open a network connection
 Open a file to read/write

Tear-Down
 Tasks to clean up after execution of each
test case.
 Ensures
 Resources are released
 the system is in a known state for the next test
case
 Clean up should not be done at the end of a
test case,
 since a failure ends execution of a test case at
that point

Method Annotations for Set-Up and Tear-Down
 @Before annotation: set-up
 code to run before each test case.
 @After annotation: Teardown
 code to run after each test case.
 will run regardless of the verdict, even if exceptions are
thrown in the test case or an assertion fails.
 Multiple annotations are allowed
 all methods annotated with @Before will be run before
each test case
 but no guarantee of execution order

Example: Using a File as a Test Fixture
public class OutputTest { @Test
private File output; public void test1WithFile() {
@Before // code for test case
public void createOutputFile() { …
output = new File(...); }
}
@After @Test
public void deleteOutputFile() { public void test2WithFile() {
output.close();
// code for test case
output.delete();
} …
}
}

Method Execution Order
1. createOutputFile()
2. test1WithFile()
3. deleteOutputFile()
4. createOutputFile()
5. test2WithFile()
6. deleteOutputFile()
Not guaranteed:
test1WithFile runs before test2WithFile

Once-Only Set-Up
 @BeforeClass annotation on a static method
 one method only
 Run the method once only for the entire test class
 before any of the tests, and
 before any @Before method(s)
 Useful for starting servers, opening connections,

etc.
 No need to reset/restart for each test case
 Shared, non-destructive
@BeforeClass
public static void anyName() {
// class setup code here
}
Once-Only Tear-Down
 @AfterClass annotation on a static method
 one method only
 Run the method once only for the entire test
class
 after any of the tests
 after any @After method(s)
 Useful for stopping servers, closing connections,
etc.
@AfterClass
public static void anyName() {
// class clean up code here
}
Timed Tests
 Useful for simple performance test
 Network communication
 Complex computation
 The timeout parameter of @Test annotation
 in milliseconds
@Test(timeout=5000)
public void testLengthyOperation() {
...
}
 The test fails
 if timeout occurs before the test method completes

Parameterized Tests
 Repeat a test case multiple times with different data
 Define a parameterized test
 Class annotation, defines a test runner
@RunWith(Parameterized.class)
 Define a constructor
» Input and expected output values for one data point
 Define a static method returns a Collection of data
points
» Annotation @Parameter [or @Parameters,
depending]
» Each data point:
an array, whose elements match the constructor
arguments
Running a Parameterized Test
 Use a parameterized test runner
 For each data point provided by the
parameter method
 Construct an instance of the class with the data
point
 Execute all test methods defined in the class

Parameterized Test Example – Program Under Test
public class Calculator {

public long factorial(int n) {
…
return result;
}
}
See Junit2.zip

Parameterized Test Example – The Test Class
@RunWith(Parameterized.class)
public class CalculatorTest {
private long expected; // expected output

private int value; // input value
public CalculatorTest(long expected, int value) {

this.expected = expected;
this.value = value;
}

Parameterized Tests Example – The Parameter Method
@Parameters
public static Collection<Integer[]> data() {
return Arrays.asList(new Integer[][] {
{ 1, 0 }, // expected, value
{ 1, 1 },
{ 2, 2 },
{ 24, 4 },
{ 5040, 7 }, });
}

Parameterized Tests Example – The Test Method
private long expected; // expected output

private int value; // input value
….
@Test
public void factorialTest() {
Calculator calc = new Calculator();
assertEquals(expected, calc.factorial(value));
}

Readings and References
 Chapter 10 of the textbook.
 JUnit documentation
http://junit.org

 An example of parameterized test

 JUnit2.zip in D2L

Next Class
Topic:
Black Box Testing Part 2, JUnit & Ant
Reading:
Chapter 10 of the textbook.
Articles on the class page and reading list
Assignment 4 – Parameterized Test
 Due April 25, 2017
Assignment 5 – Black Box Testing – Part 1: Test Case
Design
 Due May 2, 2017

SE 433/333 Software Testing & Quality Assurance

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SE 433/333 Software Testing & Quality Assurance

Uploaded by

Copyright:

Available Formats

SE 433/333 Software Testing

& Quality Assurance

Dennis Mumaugh, Instructor

April 18, 2017 SE 433: Lecture 4 1 of 101

April 18, 2017 SE 433: Lecture 4 2 of 101

April 18, 2017 SE 433: Lecture 4 3 of 101

April 18, 2017 SE 433: Lecture 4 4 of 101

April 18, 2017 SE 433: Lecture 4 5 of 101

April 18, 2017 SE 433: Lecture 4 6 of 101

 Handles illegal input -1

April 18, 2017 SE 433: Lecture 4 7 of 101

April 18, 2017 SE 433: Lecture 4 8 of 101

April 18, 2017 SE 433: Lecture 4 9 of 101

April 18, 2017 SE 433: Lecture 4 10 of 101

 The Knight Capital Group is an American global financial

April 18, 2017 SE 433: Lecture 4 11 of 101

April 18, 2017 SE 433: Lecture 4 12 of 101

April 18, 2017 SE 433: Lecture 4 13 of 101

 SEC launched an investigation in Nov 2012.

April 18, 2017 SE 433: Lecture 4 15 of 101

April 18, 2017 SE 433: Lecture 4 16 of 101

April 18, 2017 SE 433: Lecture 4 17 of 101

April 18, 2017 SE 433: Lecture 4 18 of 101

April 18, 2017 SE 433: Lecture 4 19 of 101

April 18, 2017 SE 433: Lecture 4 20 of 101

April 18, 2017 SE 433: Lecture 4 21 of 101

April 18, 2017 SE 433: Lecture 4 22 of 101

April 18, 2017 SE 433: Lecture 4 23 of 101

April 18, 2017 SE 433: Lecture 4 24 of 101

 Also known as:

 Functional specification = description of intended

April 18, 2017 SE 433: Lecture 4 25 of 101

April 18, 2017 SE 433: Lecture 4 26 of 101

 Defect: incomplete implementation logic

April 18, 2017 SE 433: Lecture 4 27 of 101

If we systematically test some Functional testing is one way of

April 18, 2017 SE 433: Lecture 4 29 of 101

April 18, 2017 SE 433: Lecture 4 30 of 101

April 18, 2017 SE 433: Lecture 4 31 of 101

April 18, 2017 SE 433: Lecture 4 32 of 101

April 18, 2017 SE 433: Lecture 4 33 of 101

April 18, 2017 SE 433: Lecture 4 34 of 101

April 18, 2017 SE 433: Lecture 4 35 of 101

April 18, 2017 SE 433: Lecture 4 36 of 101

April 18, 2017 SE 433: Lecture 4 37 of 101

 Input: ZIP code (5-digit US Postal

April 18, 2017 SE 433: Lecture 4 38 of 101

Simple example with

 Correct zip code

April 18, 2017 SE 433: Lecture 4 39 of 101

April 18, 2017 SE 433: Lecture 4 40 of 101

April 18, 2017 SE 433: Lecture 4 41 of 101

Failures are rarely the result of the

April 18, 2017 SE 433: Lecture 4 42 of 101

April 18, 2017 SE 433: Lecture 4 43 of 101

April 18, 2017 SE 433: Lecture 4 44 of 101

April 18, 2017 SE 433: Lecture 4 45 of 101

April 18, 2017 SE 433: Lecture 4 46 of 101

April 18, 2017 SE 433: Lecture 4 47 of 101

April 18, 2017 SE 433: Lecture 4 48 of 101

April 18, 2017 SE 433: Lecture 4 49 of 101

April 18, 2017 SE 433: Lecture 4 50 of 101

April 18, 2017 SE 433: Lecture 4 51 of 101