Automated Testing in an Agile

Environment
Presented by Ken Arneson
Developed by Bob Crews and Ken Arneson
Checkpoint Technologies, Inc.
 About Checkpoint Technologies
 Checkpoint Technologies, headquartered in Tampa,
FL, is a solutions provider which focuses strictly on
Quality Assurance and Software Testing with an
emphasis on Functional, Performance, and Security
testing.

 We provide leading edge software as VARs as well

as consulting, training, mentoring, and staff
augmentation.

 We are an HP Reselling Partner and Certified

Training Partner with a new training facility located
in St. Petersburg, FL.
 What is Agile?
 In the late 1990’s several development
methodologies began to get increasing public
attention.
 They all emphasized
 Close collaboration between the programmer
team and business experts
 Frequent delivery of new deployable business
value
 Tight, self-organizing teams
 Ways to craft the code and the team such that
the inevitable requirements (documentation)
churn was not a crisis.
 Agile Development
Methodologies
 Extreme Programming (XP)
 Crystal
 Adaptive Software Development (ASD)
 Scrum
 Feature Driven Development (FDD)
 Dynamic Systems Development Method
 (DSDM)
 XBreed
 The Agile Manifesto
“We are uncovering better ways of developing
software by doing it and helping others do it.

Through this work we have come to value:

 Individuals and interactions over processes and tools
 Working software over comprehensive
documentation
 Customer collaboration over contract negotiation
 Responding to change over following a plan”

SOURCE: [Link]
 What is Agile Testing?
 It’s a testing practice
 that adheres to the “Agile Manifesto”
treating the development team as the
customer of testing
 for projects using agile methodologies
 Testing in Agile
 While Agile was a methodology that
was created from a development point
of view, testing was not ignored
 The majority of the focus, however,
was placed on unit testing and
exploratory testing using the assembled
team
 Some Agile proponents did not see (or
argued against) the need for testing
specialists on an assembled team
 A Few Years Later…
 Most organizations implementing Agile
soon came to value and see the need
for a specialized testing skillset
 Testers are now almost always included
in the cross-functional Agile teams
 In some instances, these testers will
also take on non-testing tasks when
cross-skill set participation is
encouraged
 Why Testing Matters in Agile
 Testing is a discipline, even developers
following methodologies like “test driven
development” often lack good testing skills
 The success or failure of code to meet
requirements or user stories must still be
executed and documented in an auditable
fashion
 Even with business value and cake slices
being added in small increments, code still
needs to be regression tested
 Some Agile Practices
 Test-First Programming: Developers write
unit tests before coding
 Acceptance Testing: Acceptance tests are
used to verify the completion of user stories
 Short Iterations and Releases: Quick “rapid-
fire” development and testing
 “User Stories”: Short descriptions of features
that need to be coded and tested

SOURCE: Bret Pettichord 2002 “Agile Testing. What is it? Can it work?”
 More on User Stories
 A user story is a very high-level definition
of a requirement
 Just enough information for developers
and testers to produce a reasonable
estimate of the effort to implement/test it.
 Small and can be implemented quickly
 Often written on a paper medium (index
cards, post-its, etc.)
 User Story Examples
 Customers can book flights online.
 Air fare can be paid online via credit
cards.
 Customers can print boarding pass
online.
 Ticket agent can update customers
reservation via terminal.
 Gate agent can modify customers seat
assignment via terminal.
 User Story Card
367: Customer can choose seat assignment online.

Stakeholder: Chuck Connors

Tester: Mark McCain
Priority: 3
Estimate: 4
 Automated Testing In Agile
 Automated testing still fills an important
role in the world of Agile Development
 The benefits of automation still apply:
repeatability, consistency, better
coverage through data driving
 There is also the regression testing that
will be required as an iterative
approach is developed
Issues Implementing Automation
in Agile
 Automation is a development exercise, and therefore
requires planning and time to develop

 Agile projects or “iterations” are often less than a month

in length (less planning and even less time)

 Agile Developers often face the same issues when they

try to determine how much architecture and design to
do up front

 Agile projects are often developed based on a “system

metaphor” or very loose design of the product
architecture which evolves through iterations (solid rqts
are not considered essential “up-front”)
 The System Metaphor and
Automated Testing
 When Agile teams are developing the object
model or system metaphor, automated
testers should be using the same time to
develop an “automated system testing
metaphor” which will be used to create a
design approach to test scripts so they can
later be put together for regression
 As iterations are planned, time for
development of underlying infrastructure for
automated tests should be included in the
stories
 “Quick Hit” Automation
 One way automated testers can add
immediate benefit to an Agile project is to
look for opportunities for quick hit tasks
 These tasks may include: data creation
scripts, scripts which expand on developers’
test driven development examples and
expand the testing through data driven
techniques
 Any other task, which will be repeated, that
looks quickly automatable
 A Philosophy of Reacting
 Agile is built on a philosophy of being able to
quickly adapt and react to changes in
requirements
 Automated test scripts should be developed
with the same thoughts in mind
 Automated testers should look for ways to
abstract code into functions and external
data sources to allow for quick changes to
the execution steps and data
 Data drive expected results as well as input
and output data
 Use the Stories – Negative
Testing
 Often when stories are developed in Agile,
they are written with acceptance criteria
 These criteria are often simple and based on
the language of the original story
 Sometimes lost in the Agile mix are the
negative scenarios that testers conduct every
day
 Having an underlying data-driven framework
that can react to changes in interface and
requirements can also allow for injection of
negative data and highlight these issues
 Regression Testing
 Agile development teams need to
communicate and coordinate when building a
system
 Automated testers on the teams should
coordinate as well, to agree on design
principles which allow for scripts to be
integrated for regression testing
 Regression testing should take place in the
integrated application environment and run
as “cake slices” are pushed into production
 Agile versus Documentation
 Agile often eschews documentation,
but test assets (test cases, test data,
test results, defect reports, etc.) are a
large part of audits and required for
liability reasons (CMMI compliance)
 Automated test repositories can be
customized to help capture and store
these assets and consume less
enterprise time and resources while still
allowing for regulatory satisfaction
 Quality Center and Agile
 Quality Center TestDirector (or any test
management solution) can be utilized for
Agile projects
 Define “user stories” as:
 Requirements
 Test Cases
 Keep high-level and small (don’t lose the
“rapid-fire” benefits of Agile)
 Use customization features of solution to add
user-defined fields for stakeholders,
estimate, priority, etc
 Benefits
 Reusability
 Traceability
 Requirements → Tests → Execution → Defects
 Impact analysis – If a user story
changes how many tests will that affect
and what’s the estimate to implement
it?
 Audit trail (for CMMI compliance)
 Metrics & Analysis
 Summary
 Agile Development – It’s all about individuals,
working software, customer collaboration,
and responding to change (rapidly)
 Agile evolved over the years from “we don’t
need testing specialists” to “OK – maybe we
do” and, thus, Agile Testing came to be
 User stories are an important element of
Agile
 There is a place for automation in Agile (the
benefits still exist)
 Test planning tools can be utilized with the
Agile approach
Questions?
 Thank you!
Ken Arneson
Checkpoint Technologies, Inc.
Email: karneson@[Link]
Cell: (727) 612-8628