Professional Documents
Culture Documents
Agenda
Introduction
Effective Automation
Fundamental Approach
Real Facts & Think Factor
Considerations for Automation
Automation Start-up Strategies to Avoid
Guidelines & Approach
Automation Methodologies
Overview on Automation Tools
References
2 IBM Confidential
2 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Introduction
GUI automation is always viewed as the SILVER BULLET, which will save money,
time and resources
Test automation raises our hopes, yet often frustrates & disappoints us
Promises to deliver us from a tough situation, yet can create as many problems as
it solves
3 IBM Confidential
3 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Effective Automation
4 IBM Confidential
4 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Fundamental Approach
"Learning from others is the most effective way to grow your automation expertise"
5 IBM Confidential
5 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Real Fact
Automation is initially expensive and it may not do a better job than manual testing
in some situations
Think Factors
1. How much will automating and running a test cost, versus running it manually?
2. Automated test has a limited lifetime in which it must recoup the additional cost.
Will this test be reusable long enough? What factors will prevent it from being
reusable?
3. The automated test will find bugs it was designed to test for and in addition it will
find bugs that are not directly related to the test. What is the likelihood that this test
will find additional bugs?
6 IBM Confidential
6 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Real Fact
Automation is initially expensive and it may not do a better job than manual testing
in some situations
Think Factors
1. How much will automating and running a test cost, versus running it manually?
2. Automated test has a limited lifetime in which it must recoup the additional cost.
Will this test be reusable long enough? What factors will prevent it from being
reusable?
3. The automated test will find bugs it was designed to test for and in addition it will
find bugs that are not directly related to the test. What is the likelihood that this test
will find additional bugs?
7 IBM Confidential
7 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
8 IBM Confidential
8 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Record / Playback initially using the Testing tool’s wizard or toolbar will be easier but
hard to maintain because they represent not only WHAT needs to be tested, but
HOW it is to be tested
Guideline: 1
Once familiarity in scripting is achieved, continue using the scripting language of an
automation tool to provide your own implementation logic (framework) that is separate
from your test case scripts
Requires more investment upfront, but the code will be less expensive to maintain
over the life span of multiple product releases
Example:
Imagine having recorded test case scripts for a banking application that initially
allowed the user to simply click "OK" to commit a sensitive transaction, then the
application developers changed the navigational logic requiring the user to type "yes"
to confirm these transactions. Every recorded test case that has hard coded "OK"
clicks relating to the sensitive transaction commits would require maintenance.
9 IBM Confidential
9 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 2
Establish complete Automation plan, Structure, and Standards
10 IBM Confidential
10 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 3
“Separating the What from the How”
11 IBM Confidential
11 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 4
Use the 6 phases for implementing the best course of Automated Testing action
- Evaluation
- Planning
- Design
- Implementation
- Deployment
- Review
12 IBM Confidential
12 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 5
A. Define Automation Goals & Prioritize
At every point in the specification process, you can check the "testability" of the requirement or
specification by ensuring it is S.M.A.R.T.
S - Specific
M - Measurable
A - Attainable or Achievable
R - Realistic
T - Timely
Guideline: 6
Guideline: 7
14 IBM Confidential
14 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 8
Define Design Goals & Prioritize
- Maintainability
- Structured
- Well Documented
- Reviewed
- Framework Specific Goals
- Modularity
- Reusability
- Usability
- Scalability
- Reusability
- Portability
- Robustness
- Reliability
- Efficiency
- Flexibility
15 IBM Confidential
15 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Guideline: 9
Determine the Automation Interface Architecture
- Data Driven Interface
- Programming Interface
Guideline: 10
Determine the Automation Framework Architecture
- Error Recovery
- Support of Custom Controls
- Product Testability Issues
- NLS – Localization
Guideline: 11
Define Standards – Naming & Code Conventions
16 IBM Confidential
16 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Interface Documents
Automation Developers
Framework
17 IBM Confidential
17 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Automation Methodologies
18 IBM Confidential
18 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
Most of the Automated Tools in the market remain as an integrated suite for Test
Documentations, Scripting, Executions & Defect Tracking
Automation Tools available in the market which are more on focus today includes
Rational Tools – Robot, Test Manager, ClearQuest
Mercury Interactive – WinRunner, LoadRunner, QTP, Astra Load Test
Segue – Silk Performer
19 IBM Confidential
19 © Copyright IBM Corporation 2005
IBM Global
IBM Global
Services
Services
– Testing Competency
References
Bibliography
1. Dustin, Elfriede, Rashka, Jeff, and Paul, John (1999) Automated Software Testing:
Introduction, Management and Performance. Addison-Wesley Longman, Inc. Reading,
Massachusetts
2. Fewster, Mark and Graham, Dorothy (1999) Software Test Automation: Effective
use of test execution tools. Addison-Wesley: ACM Press, New York
3. Kaner, Cem, Falk, Jack, and Nguyen, Hung Quoc (1999) Testing Computer
Software. Wiley Computer Publishing, New York
20 IBM Confidential
20 © Copyright IBM Corporation 2005
IBM Global Services – Testing Competency