Unit 4

Test Management
 Perception and misconceptions about testing

•Perception 1: Testing is not technically challenging

•Testing is a simple and repetitive job
•It does not require specialized skills
•It is manual task
•If product is simple, it is easy to test
•Misconception:
•Testing needs
• understanding of entire project
•Domain expert
•Understanding of multiple domains
•Specialized in programming and script language
•Perception 2: Testing does not provide me a career path or growth
•Career path and rules defined for the each organization people differently
•It helps to improve development, standards, different software engineering
functions
•Misconception:
•No much career path opportunities in testing flied
•Define separate rules for development engineer, senior development engineer,
business analyst, domain expert,etc
•Perception 3: I am put in testing – what is wrong with me?
•Testing is an activity that is closest to releasing the product to customers
•Testers don’t associated with other job functions or organization people with
gleam messages.
•These messages convey from others and spread over the organization
•Perception 4: These folks are my adversaries
•Testing and development teams should reinforce each other and not be at logger
heads
•As the main function of testing is top find errors in the software, it is easy to the
adversary attitude to develop between testing and development team
•Perception 5: Testing is what I can do in the end if I get time
•Testing is not what happens in the end of project. It happens through out and
continue even beyond the release
•Testing is a planned one. It is not constructed end of the cycle
•Adequate time, resources, software, knowledge are essential for testing
•Perception 6: there is no sense of ownership in testing
•Testing has deliverables just as development has and hence testers should have
the same sense of ownership.
•Module leader is referred to ownership of the module which has been
development
•In the maintenance phase, there are two ownership in that level(i.e module level
ownership and component level ownership)
•Perception 7: Testing is only destructive
•Testing is destructive as much it is constructive, like the two sides of a coin
•Tester must perform/ destructive,
•The part which does not work
•What works going on the product
• express the work in the product
•Analyze the risk of the product before release
II comparison between testing and development
1. Testing is frequently said to be a crunch time function
• Testing is treated carefully in the product release time
• Because it throws some unique planning, management challenge,
deliverables, etc
2. More “elasticity” is allowed in project’s in early phases
Planning testing projects affords less flexibility than development projects
3. Testing function are arguably the most difficult one to staff
• Testing is not a smaller function. Only minimum number of people choose
the testing as job
• Because it is hard to attract and retain top talent for testing functions and its
also carried under time pressure
4. Testing function hold several external dependencies when compared with the
development functions
Reasons,
• Testing is done at the end of the project life cycle
• Testing activities reach to a standstill until particular defects are fixed
III providing carrier paths for testing professionals
When performance carrier path testing, must noted following areas
1. Technical challenge
2. Learning opportunities
3. Increasing responsibility
4. Increasing authority
5. Increasing independence
6. Ability of an organization success
7. Rewards, etc
3 stages of individual in testing group goes through
8. Follow stage
9. Formulation stage
10. Test lead stage
Follow stage:
It is beginning stage of trust and confidence level between individual and
organization
Following instruction must convey,
1. What task to be done?
2. How those task to be done?
These instruction are necessary for both people(individual and organization)
Follow stage works are,
3. It is fairly task oriented
4. Given with detailed instruction
5. Leaves little requirement for decision making
Formulation stage :
In this stage of carrier, to focus on
6. Independent decision making
7. Clarity in communication to assign work for others
8. interacting with other groups
Individual perform,
9. To delegate and monitor the work
10. To plan ahead
Influence/ test lead stage:
Each individual acts as a role model
• The people in an organization may refer the role model to get advise and
guidance for the chosen tasks
• The role models also posses additional responsibilities in different
dimensions.
Technical growth management growth

Test architect Department

head manager

Test lead, senior

test engineer, test
engineer
IV. The role of Ecosystem and a call for action role of education system
Education system does not provide sufficient importance in the testing field.
Reasons are,
• Many formal core courses on programming are there. But only few
universities offer core courses on software testing.
• Only few lab courses for common testing tools, compare to other
programming languages
• No reward for test engineers and their skills
List of testing awareness:
• To create a course on software testing for all students Eg: CS/IT/SE
• The right values can only be more effectively caught by the students than be
taught by the teachers
Role of senior management:
• Senior management of organization plays a vital role in development of
testing people
• Senior management people commitment has to be must translated into
“visible action”
 Organization structures for testing teams
Dimensions of organization structure:
• Organization structure gives a road map for team members to take over their
career paths
• Organization structures are viewing upon 2 dimensions.

Organization structure

1. Organization type 2. Geographic

distribution

1. Product 2. Service 3. single-site 4. Multi site

organization organization
1. Organization type:
1. Product organizations
• Software product are development here
• Testing is done by one of groups
• Software projects are not handled
2. Service organizations
• Does not handle the entire product
• They provide testing services to other organizations
• These organization are specialist for testing services only
2. Geographic distribution
1. Single site: all team members are located and worked in one place
2. Multi site : entire team is spreaded and worked across many locations
Structures in single product companies:
• Product companies in general have a high level organization structure
 CEO

CTO Product I Product 2

business business Support
unit unit
Product
management Product
management

Product delivery
Product delivery
CTO: fix the high level technology directions for concern
Product business unit: it holds responsibility of product management group and
product delivery group
Product management group: it produces road map by merging CTO’s directions
and specific market needs
Product delivery group : they responsible for delivery the product and handling
both development and testing functions
Project manager: also called “development manger” or “project delivery manager ”

Testing team structures for single-product companies

• Most product companies start with a single product
• The product delivery members distribute their time among multiple tasks
• Every engineers report into project manger
• Project manager handles each and every part of the projects
 Project manager

Teams/ engineer Teams /engineer Teams/ engineer

Develop

Test Test

Develop
Component wise testing teams
• The product is development by using the components
• Every component is developed and tested by separate teams
• All these components are integrated by using a single integration test team
• This team report to the project manger
Project manager

Component manager Component manager Integration manager

Development
Develop
team
Test Test
Test team
Develop
Structure for multi product companies:
Every product is treated as separate business unit
Organization of test teams in multi product companies is dictated largely by the
following factors
1. How tightly coupled the product are in terms of technology
2. Dependence among various products
3. How synchronous are release cycle of products
4. Customer is base for every product and similarity among customer bases for
various products
Testing teams as part of CTOs office:
• Development and testing are the same level of importance in the concern
• Testing team report directly to CTO as a part to design and development teams.
Advantages:
• Developing a product architecture that is testable or suitable for testing.
• Testing team will have better product and technology skills
• Cost effective strategy for test automation
Single test team for all products
• Single team is responsible for all type of products
• It is divided into multiple components
• Every component is developed by an independent team
Testing team organized by product
• It is based on accountability, decision making and scheduling
• Complete responsibility is assigned to testing team
• Unit head organized the testing and development team
Separate testing teams for different phases of testing:
• It is done by several types of testing and different levels of skills
• Types of testing: black box, white box, regression testing
• levels of skills- test script, test case, test report
Testing services for organizations:
• Business needs
• Increasingly specialized function
• Give variety and complexity function
• Better out sourcing
• Understanding domain
• More economical/ cost factors
Typical roles and responsibilities of testing service organization
1. Account manager:
• He makes a contact between customer and the testing services organizations
• Makes a strategic communication between customer and the testing services
2. Near site team:
• It is small size team
• It is kept near to customer site
• It is used to point of contact for the customer to the urgent issues
• Fulfill the gap between customers and service organization
3. Remote team:
• Team placed on testing service in organization site
• It is a large size team (perform lot of work)
• Remote team based on,
1. Size of project
2. Complexity of project
• They perform peer to peer communication, managing work, etc
• Both team can be rotated around
• The remote team member can visit the customer site for any onsite testing
Success factors for testing organization:
• Communication and team work
• Bringing in customer perspective
• Providing appropriate tools and environment
• Providing periodic skill upgrades
 CEO/VP

Customer contact Account manager

Remote team manager

Near site team

Test leads

Test engineer
Challenging and issues of testing services organization
1. Outsider effect and estimation of resources
1. Not possible to know product history for them
2. Internal development and testing team do not necessary to have information
about software and hardware resources required.
2. Domain expertise
1. Eg: ERP domain, logistic domain, banking domain, etc
3. Privacy and customer isolation issues
1. Physical isolation of different team is difficult. Because the testing services
organization have a common infrastructure
4. Apportioning hardware and software resources and costs
1. When testing service organization involved in project, it uses hardware and
software resources.
2. It helps to identify other resources also. They are links, servers,
infrastructure, costs, etc
5. Maintaining a bench:
1. To keep a people on bench
2. Customer suddenly give a new project to the testing services organizations
Test planning:
• Testing should be driven by a plan
• Test plan acts as the anchor for the execution, tracking and reporting of the entire
testing projects
• It covers,
1. Overall test objective
2. What to test
3. Who will test
4. How to test
5. When to test
6. When to stop testing
Scope management:
• It pertains to specify the scope of a project
• Understand what constitutes a release of a project
• Breaking down release into features
• Prioritizing the features for testing
• Deciding which features will be tested and which will not be.
Deciding test approach/ strategy:
• Testing strategy components are,
• Scope & objectives
• Business issues (budget)
• Testing approach
• Roles and responsibilities
• Test deliverables
• Communication and status reporting
• Define reporting and tracking
• Testing measurement and metrics
• Risks and mitigation
• Training plan
• Change and configuration management
• Test automation and tools
Risk management
• Risk are events, it potentially affect a project out comes
• It must be controlled by project manager
• Project manager performs
1. Identifying the possible risks
2. Quantifying the risks
3. Find non technical problems
4. Planning how to solve the risks
5. Responding the risks

Risk Risk
identification quantification

Risk mitigation
Risk response and planning
Test process:

Putting together and base lining a test plan

Test case specification

Update traceability matrix

Identify possible candidates for automation

Developing an dbase lining test cases

Executing test cases and keeping traceability matrix

Collecting and analyzing matrics

Putting together and base lining a test plan:
• Test plan is an anchor point for every project
• Every test plan is designed based on template
• It is reviewed by designated people in an a organization
• A complete authority later approved it
• After test plan is base lined into configuration management repository
Test case specification:
• Test case creation depends on
• Test purpose
• Items to be tested
• Environment to run the test cases
• Input data to be given
• Execution steps
Update traceability matrix:
• It helps to verify every requirement is tested or not
• It is developed in requirement gathering phase
Identifying possible candidates for automation
It helps to decide,
• Which test case to be automated?
• To analyze the required automation tools, human beings
Developing and base lining test cases
• Creation of test cases entails translating the test specifications to a form from
which test can be executed
Executing test cases and keeping traceability matrix
• Test case to be executed at correct time
• Eg: done smoke/ sanity test
Collecting and analyzing metrics
• Basic measurement form running the tests are converted to meaningful metric
by use of suitable transformations
Reporting test results:
Document that records data obtained from an experiment of evaluation in
an organized manner, describe the environmental or operating conditions, and
shows the comparison of test results with test objectives
Some of test report are,
 Test log report
 Test summary report
 Test incident report
 Test status report, etc
Test log report
• It is a document which contains all information about the test results, and it
contains the expected results also
• It captures raw output during a unique execution of one or more tests
 Test log report template

Test log identifier

it is unique number
Description
items to be tested
version number
associated test item
Activity and events entries
Dates and names of test log authors
Execution description
Procedure results
Environmental information
Test incident report
• Also called as bug report
• Test incident documents all issues found during different test phases
• It is detail of actual versus excepted results of a test , when a a test has failed
and anything indicating why test failed
• Uniquely numbered and tracked to resolution
• Defect tracking tools are preferred
Test summary report:
• Prepared at end of testing projects,
• It helps to explain various details and activities about testing performed for
project
• It contains
• Purpose of document
• Application overview
• Testing scope
• Metrics
• Types of testing performed
1. Test incident report identifier
2. Summary
• References
• test procedures
•Test case specifications
•Test logs
•Other supporting materials

3. Incident description
•Expected results
• Actual result
• Anomalies
• Data and time

4. Impact
severity
mission critical
major
minor
priority
immediate
delayed
duplicate
 Role of 3 groups in test planning and policy management
3 critical groups in testing are,
• Managers
• Testers/ developers
• User/clients
In TMM they are called as 3 critical views(CV)
Manager’s view : involves commitment
Developer (testers view) : encompasses technical activities and their
activities
User/clients view: cooperating or supporting view
Developers/ testers activities are
• Working with management to develop testing and debugging policies and
goals
• Participating in teams that oversee policy compliance and change
management
• Familiarizing them selves with approved set of testing/ debugging goals
and policies
 Role of 3 groups in test planning and policy management
3 critical groups in testing are,
• Managers
• Testers/ developers
• User/clients
In TMM they are called as 3 critical views(CV)
Manager’s view : involves commitment
Developer (testers view) : encompasses technical activities and their
activities
User/clients view: cooperating or supporting view
Developers/ testers activities are
• Working with management to develop testing and debugging policies and
goals
• Participating in teams that oversee policy compliance and change
management
• Familiarizing them selves with approved set of testing/ debugging goals
and policies
 Managers Developers/ tester Users/ clients

•Apply BBT/WBT
•Specify requirements
•Task forces, policies, •Assist with test
clearly
standards planning
•Support operational
•Planning •Test at all levels
profile
•Resource allocation •Train at all levels
•Participate usability
•Support •Train and mentor
test
training/education •Participate in task
•Participate
•Interact with user/ force
acceptance test
clients •Interact with user/
planning
clients

Achievement of testing goals

 Introduction test specialist
• Organization tests its software at several levels(unit, integration, system, etc)
• More than TMM level 2 companies, must maintain test group of organization.
• Test group must perform all types of testing
• Test group of an organization acquires leadership in areas that relate to testing
and quality issues
• Staffs members of testing group are called “test specialist” or test engineers
• Testers are not developers/ analyst
• Testers don’t repair code
• They add value to software product
• They are not destructive, they are constructive
 SKILLS NEEDED BY A TEST SPECIALIST
Personal and managerial skills
• Organizational and planning skills
• Ability to track and attention to detail s
• Determination to discover and solve problems
• Ability to work with others
• Ability to resolve conflict
• Ability to mentor and train others
Technical skills
• General software engineering principles and practices
• Understanding of testing principles and practices
• Understanding of basic testing strategies and methods
• Ability to plan, design and execute test cases
• Knowledge of process issues
• Knowledge of configuration management
• Knowledge of test related documents
• Ability to define, collect and analyze test measurement
 Building a testing groups
Step 1: upper management support for test
function

Step 2: establish test group organization, carrier

paths

Step3: define education and skill levels

Step 4: develop the job description

Step5: interview candidates

Step 6: select test group members

• Organizing and staffing and directing were major activities required to
manage a project and process
Staffing activities,
• Billing positions
• Assimilating new personal
• Education and training
• Staff evaluation
Directing includes,
• Providing leadership
• Building terms
• Facilitating communication
• Motivating personnel
• Resolving conflicts
• Delegating authority
Organizing includes,
• Selecting organizational structures
• Creating positions
• Defining responsibilities
Step1 :
• Establish a test group and commit resources to group.
• How the testing group will be organized
• How group fits into organizational structure
Step2:
• Hiring staff to fill test specialist position , management should have a clear
idea
• Develop testing group will be organized?
• How group fits into organizational structure
Step 4:
• Job title
• Full time/part time
• Location
• Salary
• Qualification
Step 5:
• It should be a structural and of a problem solving nature
• Interviewer prepare extensive list of questions with technical background
Step 6:
• Test group member has been selected based on projects, knowledge, skills
• Based on performance of test group member, they may be retain/likely to
retain/likely to release/release