Professional Documents
Culture Documents
2
Introduction – Topics to Cover
OHT 10.3
3
Ultimate Desired Outcomes for the
OHT 10.4
Chapter
• Today:
– Describe the process of planning and designing tests
– Discuss the sources for test cases, with their advantages
and disadvantages
• Next:
– List the main types of automated software tests
– Discuss the advantages and disadvantages of automated
computerized testing as compared to manual testing
– Explain alpha and beta site test implementation and
discuss their advantages and disadvantages.
4
OHT 10.5
10.1 The Testing Process
• Testing is done throughout the development process.
• Testing is divided into phases beginning in the
design phase and ending at the customer’s site.
5
OHT 10.6
6
OHT 10.7
Determining the Appropriate
Software Quality Standard
• Different standards required for different software applications.
7
OHT 10.8
1. Financial losses
* Damages paid for physical injuries
Aircraft or auto instrumentation; health equipment…. Law suites!!
* Damages paid to organizations for malfunctioning of
software
Companies have many lawyers on staff!!!
* Purchase cost reimbursed to customers
* High maintenance expenses for repair of failed systems
2. Non-quantitative damages
* Expected to affect future sales
* Substantially reduced current sales
9
OHT 10.10
Determining Software Testing Strategy
• Big Bank or Incremental? So, do we want the
testing strategy to be big bang or incremental?
– Major testing at end in the past….
– If incremental, top down or bottom up?
11
OHT 10.12
Lots of Questions
• So we first need to consider five basic issues:
– What to test
13
OHT 10.14
14
OHT 10.15
Rating Units, Integrations, and
Applications
• We need to rate these issues to determine their
priority in the testing plan.
• Rate based on two factors:
• 1. Damage severity level – severity of results if
module / application fails.
– How much damage is done??
– Will it destroy our business? Our reputation??
• 2. Software risk level – what is the probability of
failure.
Module/Application Issues
1. Magnitude (size)
2. Complexity and difficulty
3. Percentage of original software (vs. percentage of
reused software)
Programmer Issues
4. Professional qualifications
5. Experience with the module's specific subject matter.
6. Availability of professional support (backup of
knowledgeable and experience).
7. Acquaintance with the programmer and the ability to
evaluate his/her capabilities.
16
OHT 10.17 Computations
• Essential to calculate risk levels
– We must budget our testing due to high cost
– But we must temper our testing with cost/risk!
17
OHT 10.18
• If we are including unit, integration, and applications in a
test plan, we need to know how much/many resources are
needed.
– Thus, we need to prioritize.
• Higher the rating, and priority more allocation of
testing resources are needed.
• Consider:
– Some tests are based on high percentages of reused code.
– Some applications are developed by new employees
• We typically use a 5 point scale, where 5 is high.
• (Variations include a 1-5 severity level and a probability of
0.0 to 1.0 of their occurrence.)
• Can see results for Super Teacher application in next slide.
18
OHT 10.19
19
1.
OHT 10.20
Which Sources Should be Used for
Test Cases?
• Do we use live test cases or synthetic test cases.
• All three types of tests should consider these.
20
OHT 10.21
Who Performs the Tests?
• Unit Testing – done by the programmer and/or development
team.
21
OHT 10.22
Where to Perform the Tests?
• Typically at the software developer’s site..
22
OHT 10.23
When are Tests Terminated?
• This is always the $64,000 question!!
23
OHT 10.24
24
OHT 10.25
25
OHT 10.26
26
OHT 10.27
• This means stop when budgets or time for testing has run out.
27
OHT 10.28
Test Plan
• Lastly, system testing is documented in a software
test plan.
28
OHT 10.29
29
OHT 10.30
30
OHT 10.31
3. Testing process
3.1 Instructions for input, detailing every step of the input process
3.2 Data to be recorded during the tests
34
OHT 10.35
Regression Testing
• Need not test everything.
• Typically re-test only those artifacts directly changed
and those providing inputs and outputs to these
changed artifacts (modules).
• Very often new errors are introduced when changes
are made.
35
OHT 10.36
36
OHT 10.37
37
OHT 10.38
3. Test results
3.1 Test identification
3.2 Test case results (for each test case individually)
38