White Paper

Testing in an Agile
Environment
Last Updated: 15th January, 2009
Introduction
The principal indicators of a growing business are a
widening customer base and rising return on investment
(ROI). Various parameters steer these indicators in a
positive direction, chiefly cost, time and quality. Balancing
these parameters is of paramount significance for business
growth. Companies need to deliver, cost effectively, the right
product, in the right way, to meet customer requirements
with assured quality. They must do this first time every
time to ensure that they can bring new products and
services to market ahead of their competition. Companies
also need to extensively consider market dynamics
including technological developments, changing customer
preferences, evolving global standards and regulation.
These market developments add further complexity and
force companies to enhance the frequency of their product
updates and accelerate delivery schedules.
In software product development, an Agile methodology
is one strategy adopted while addressing the challenges
of market dynamics. As customer satisfaction and ROI are
linked to product quality and cost of quality, QA and testing
services have a critical bearing on the success of any Agile
project.
This white paper explores QA and testing activities in an
Agile environment and the critical characteristics of the
testing team that ensure the success of development
projects in an Agile environment. As there is broad-based
acceptance of agile practices, an introduction to these
practices and agile development methodologies is not
provided in this paper.
Testing Agenda in Agile Projects
Agile methodologies are designed to break the software
down into manageable parts that can be delivered earlier
to the customer. The aim of any Agile project is to deliver
a basic working product as quickly as possible and then
to go through a process of continual improvement. An
Agile project is characterized by having a large number of
short delivery cycles (sprints) and priority is given to the
feedback-loops from one cycle to the next. The feedback-
loops drive continuous improvement and allow the issues
that inevitably occur (including the way requirements have
been defined or the quality of the solution provided) to be
dealt with much earlier in the development life cycle. To
achieve this, companies need to re-think their approach to
delivery and have their previously independent contributors
(Business Analysts, Developers, Testers, End Users etc.)
work together in teams.
AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page 
The testing agenda during the development phase is very
different to the traditional approaches that support the
Waterfall or V-Model development methodologies because
Agile uses a Test Driven Development (TDD) model.
However, many of the requirements of the extended testing
model will still apply.
Project Initiation Phase
For an Agile project to succeed, it requires effective
planning and management. As part of the project team
the Test Manager is responsible for establishing the quality
processes, identifying test resources and delivering the test
strategy. The test strategy will include details of the Agile
development process being used. It will also include details
of test phases not directly related to the Agile development
for example, Operational Acceptance or Performance
Testing.
Development Phase
True Agile development uses a “Test Driven Development”
(TDD) approach. The testers, developers, business analysts
and project stakeholders all contribute to kick-off meetings
where the “user stories” are selected for the next sprint.
Sprints are usually of fixed duration and selecting the
right number of stories and estimating the time to deliver
them is the key to success. The estimation process can
often be inaccurate at the beginning of an Agile project
but improves as the team’s experience grows within the
specific implementation they are working on.
Once the user stories are selected for the sprint they are
used as the basis for a set of tests. The testers create test
scenarios which are presented to the business analysts
and project stakeholders for their approval and signoff.
These test scenarios are then broken down to test cases
that offer adequate test coverage for the given functionality.
The developers then write code that will pass the tests.
In this approach the development and testing take place
continuously throughout the sprint – there is no separate
testing phase. In the TDD approach the features are tested
throughout the sprint and the results presented to the
stakeholders for immediate feedback. The test scenarios
defined are not limited to functional testing but can include
other types of testing including performance and integration
testing when the product is mature enough.
While the development is underway the user stories for the
next sprint are written. These include the stories specified
© 2009 AppLabs
in the delivery plan but will also include additional stories
required to cover any issues that have been identified as
part of the feedback process from previous sprints.
Sprints in an Agile project can extend to multiple levels in
a complex system. A sprint might not lead to a product
release if it does not add enough functionality to the
product being developed. The stakeholders take a decision
on when the application should be moved to the release
phase depending on the market need or the level of key
functionality being added to the system. While multiple
iterations may be required to release a product, there may
also be cases where releases are more regular owing to the
additional value delivered at each iteration level. Whichever
release approach is adopted, the testing team’s goal is
to have a release available with minimal defects and low
implementation risk at the end of the Sprint.
As functionality grows with each iteration, regression testing
must be performed to ensure that existing functionality has
not been impacted by the introduction of new functionality
in each iteration cycle. Defect fixes should also be followed
by extensive regression testing. The scale of the regression
testing grows with each sprint and to ensure that this
remains a manageable task the test team should use test
automation for the regression suite and focus their manual
testing effort towards locating new defects during the build
phase.
Release Phase
The aim of any Agile project is to deliver a basic working
product as quickly as possible and then to go through a
process of continual improvement. This means that there
is no single release phase for a product. The product will
move into production when enough of the functionality
is working and the quality of the product is acceptable
to the stakeholders. Prior to release, a final acceptance
test is performed before transitioning the application into
production. The testing activities listed above are not
exhaustive but broadly cover the areas which the testing
team contributes to the Agile approach.
Getting Equipped for Agile Testing
Agile projects present their own challenges to the testing
team. Unclear project scope, multiple iterations, minimal
documentation, early and frequent testing needs and active
stakeholder involvement all demand special and diverse
skills from the testing team. Some of the essential skills are
illustrated here:
AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page 
Resource Management
The Agile approach required a mixture of test skills
– usually held across one team. Test resource will be
required to define the scenarios and test cases, conduct
manual testing alongside the developers, write automated
regression tests and execute the automated regression
packs. As the project progresses, specialist skills will be
required to cover further test areas that might include
integration and performance testing. The availability of a
pool of professional test resources offers the scope for
effective resource management across the agile project
life cycle. There should be an appropriate mix of domain
specialists who plan and gather requirements in addition
to test engineers who will have multiple skill sets and own
the test execution.
Communication
The benefits of independent testing will not be realized
unless good communication exists between the testers,
developers, business analysts and project stakeholders.
The iterative model of development and the frequency
of releases mandate that all teams have a common
understanding of the user requirements. The testing teams
should be skilled in the use of change management tools.
The communication model used by the testing team itself
must enable both regular and ad-hoc project updates from
various parties engaged in the project. The testing team
should adopt the most efficient and effective methods
of conveying information to the developers, project
stakeholders, domain specialists using a combination
of face-to-face conversation, meetings and workshops,
phone calls, email and WebEx meetings.
Processes
Another key success factor for Agile Development is the
implementation of quality governance processes such
as configuration management. Typically, an organization
that has no processes in place will be in chaos. As formal
processes are implemented the organization will be able
to follow a repeatable and consistent delivery method.
Organizations considering the use of Agile should ensure
that configuration management, change management,
project management and release management are in place.
Testing teams which bring with them best practices and
are accredited with globally recognized certifications (e.g.
TMMi, CMMi, ISO etc.) will be able to help organizations
accelerate testing and enable lower cost of quality to be
achieved.
© 2009 AppLabs
Requirements Elicitation
Agile projects do not necessarily have comprehensive
documentation. Product requirements in Agile projects
are often captured in the form of user stories. The testers
should be experienced and smart in analyzing and defining
the testing requirements in the absence of any product
documentation. The ability to work side-by-side with the
developers, analysts and customers is an important part
in quickly understanding the test requirements. The testers
should also be equipped with alternative ways of deriving
test requirements. A few such approaches include:
 Getting access to all the material that can provide input
for the intended product behavior. This can include
input provided to the developer to build the application,
feature list, brief write-up on application capabilities
and partially written use cases etc.
Conclusion
Companies which adopt Agile projects should note the
importance of engaging test teams at project initiation. This
will ensure an accelerated delivery of working software. If
Agile projects are to achieve customer satisfaction and ROI
then time, cost and quality must be controlled and balanced.
To ensure accelerated delivery of working software which
conforms to the desired quality, the testing team should be
involved from the beginning of every iterative development
cycle and not just after the first couple of sprints. The
testing team must develop the necessary mindset for an
Agile project. Their own agility and flexibility is essentially
the key to their success in the project.

 Knowledge transfer from product owners and subject

matter experts (SME). Direct communication will be the
best practice here as time spent with the SME helps
to elicit more product knowledge than is expressed in
written documents.

 Gaining insights into the capabilities needed from

the product under test by exploration of comparable
products, reading user manuals and user documentation,
exploring help text via user interface and taking an
application tour through the GUI.

 The testing team performing exploratory testing at

various stages of the life cycle. The testing team
should be well trained to scope the work, ask the
right questions, and deliver a valuable output within a
few hours. Exploratory test engineers should be able
to analyze a product, think critically to evaluate risks
and craft test cases that systematically explore the
product.

 The frequency of testing is greater in an Agile project

than in a traditional development model. The testing
team should be thought leaders in test automation.
The testing team should, in conjunction with the
development team, develop an automation framework
that fits the unique requirements of the Agile project.
In addition, the testing team should be efficient in
identifying the candidates for automation and build a
suite of automated tests around the application which
enable rapid iterations.

Specialist testers from the test team should have these and
many more skills which are attained through experience
as testing professionals. Testing teams delivering services
in an Agile environment need to develop and demonstrate
these skills to contribute to the success of Agile projects.

AppLabs.com
App_WhitePaper_Testing_In_Agile_Environment_1v00 Page  © 2009 AppLabs