Agile Endgames

Releasing Agile Products
in the Enterprise
Exploring the Challenges as Agility Scales
Bob Galen
President & Principal Consultant,

RGCG, LLC – Leading you down the path of agility…
www.rgalen.com bob@rgalen.com
Introduction
Bob Galen
Somewhere ‘north’ of 20 years experience ☺
Various lifecycles – Waterfall variants, RUP, Agile, Chaos, etc.
Various domains – SaaS, Medical, Financial, Computer Systems, and
Telecommunications
Developer first, then Project Management / Leadership, then Testing
‘Pieces’ of Scrum in late 90’s; before Agile was Agile
Agility @ Lucent in 2000 – 2001 using Extreme Programming
Formally using Scrum since 2000
Most recently at ChannelAdvisor as Dir. Product Development and Agile
Architect/Coach (2007-2008)
Connect w/ me via LinkedIn if you wish…
Bias Disclaimer:
Agile is THE BEST Methodology for Software Development…
However, NOT a Silver Bullet!
Copyright © 2009 RGalen Consulting Grp,

v1.1 -- November 2008 LLC 2
Outline
Introduction
1. Agile Scaling & Scrum
2. The Evolving Role of the Product Owner
3. Criteria
4. Scrum-in-Testing
5. Testing Challenges
6. Q&A

Introduction

Scrum
We’ll be using Scrum as the Agile reference framework for

the class
Agile Project Management Framework

Roles:
Scrum Master, Product Owner & Team
Sprint – iteration (15 – 30 days)
Product Backlog – a prioritized list of: requirements, work to do, user
stories
Sprint Planning – mapping PB to tasks
Daily stand-up (15 minutes, Pigs & Chickens)
Sprint Review (demo) & Retrospective

Process Overview
Source: Adapted from Agile
Software Development with
Scrum by Ken Schwaber and
Mike Beedle.
24 hours
Daily Scrum
Meeting
Backlog tasks 30 days

expanded
Sprint Backlog by team
Potentially Shippable
Product Backlog Product Increment
As prioritized by Product Owner

Core Agility
Not always an “Enterprise Play”
Short, time boxes; typically 2 weeks
Small, co-located teams; typically 7 +/- 2
100% automated testing – run during each sprint
Continuous “Customer” collaboration
Artifact-lite; collaboration & conversation heavy
Velocity based; not plan-driven
Forecasting at a high level; scope always compromised
Change friendly
High levels of transparency (reality)

Challenge Points
Larger scale implementations: larger & distributed teams

Regulated environments with documentation requirements
Enterprise Architectural consistency
Shared team members
Subject Matter Expertise; Specialized domain expertise
Full-time doesn’t make sense
Not a prescriptive process (little guidance)
Larger scale manual testing environments (scripted)
PMO-based organizations (portfolio based decisions, earned value,
gated project commitments)
Cost-Schedule-Quality fixed; not Scope!
“Agile Customer” fully engaged; in real-time

Aspects of Enterprise Agile
There are really 4 aspects to Scaling Agility within the Enterprise:
Organizational – larger teams, distributed teams, and outsourced teams

Products – larger scale projects, interoperability, usability &
consistency, and forecasting
Development – dependencies & integration, varying processes, and
cross group collaboration
Testing & Deployment – regression, regulatory, integration, product
maturation visibility, and production deployment readiness
This section focuses on these areas from all four perspectives, with
a strong emphasis on Products & Testing

Aspects of Enterprise Agile
Industry lessons have lagged in larger scale Agile implementations

It’s not the “sweet spot” for them and we seem to be mostly on our own
In the Enterprise, Scrum is leading the way in providing guidance

towards scaling, but so far it’s sparse in nature –
Ken Schwaber – published Enterprise Scrum in 2007
Dean Leffingwell – published Scaling Software Agility in 2007
Jeff Sutherland has taken the lead in defining and sharing lessons
learned
There are still gaps from a Product Owner perspective – although the
Certified Scrum PO class tries to help
Larger scale implications have been ignored to-date – thus this

section…

Aspects of Agile Scaling Source: Mike Cohn’s
www.mountaingoatsoftware.com website.
Scrum of Scrums (of Scrums)

Meta Scrum Level – (S3)
Scrum of Scrums - (S2)
Scrum Team Level - (S1)

Siemens Medical Systems
Size Case Study
(S1) - Scrum Teams
Focused on the work
1200 developers (S2) - Scrum of Scrums

300 testers Focused on cross-team, cross
product issues & integration
Distributed across North
America, Europe, India
(S3) - Scrum of Scrums of Scrums
At any one time: 100 – 120
Focused on cross-product line
Sprints going on at once integration, compliance, PMO,
3 major product lines processes
150 Scrum Masters
Matrix - Product Owner & QA (S4) – Focused on Leadership of
organizations Scrum adoption
Agile strategy

ChannelAdvisor
Size Case Study
(S1) - Scrum Teams
Focused on the work (daily)
60-70 developers (S2) - Scrum of Scrums

15-20 testers Mostly active at integration
release points (daily, normally
Raleigh NC, Atlanta GA,
weekly)
Limerick IE, Russia
At any one time: 8-10 Sprints (S3) - Scrum of Scrums of Scrums
going on at once
Focused on staffing &
3 major product lines; 1 SaaS contention; forward looking x 2-
deployment instance 3 sprints, (on demand)
12 Scrum Masters; 6-8 Product
Owners (S4) – Focused on Leadership of
Separate Product, QA, and Scrum adoption
Development organizations Agile strategy, (on demand)

Aspects of Enterprise Scaling
Behind the Scrum of Scrums
And there are roles not defined behind the SoSoS, for example –
What are the sorts of conversations & activities that occur at each
level? Are there even levels?
Who guides the process, tools, & techniques for consistency? Or do

you even care?
How do you integrate agile processes with traditional business and

operational processes?

Aspects of Enterprise Scaling
Behind the Scrum of Scrums
What are the hierarchies behind the SoSoS?
Scrum Master(s)
Product Owner(s)
Lines of business
Team collaboration – resource management, allocation, and budgeting
Labs & equipment sharing
Reporting & release coordination
Quality, measurement, and governance
And how do they operate, integrate, and provide consistency w/o command-
and-control?

Typical S2
Charter
Coordinate amongst various Scrum teams at an integrated product level
Product integration, testing
Resource sharing & constraints (people)
Resource sharing (hardware & tools)
Operationally - Scrum dynamics
Scrum Master of the S2
Scrum Masters gather, potentially daily stand-up
Discuss cross-team progress, dependencies, impediments
Focused towards the next, integrated product release point

Managing towards short term roadmap deliverables

Typical S3
Charter
Product Backlog / Roadmap definition, 6-12+ months
Product integration, testing, and consistency checks (Compliance,
Usability, UAT)
Resource sharing & constraints (people) and (hardware & tools)
Budget / cost management
Periodic (monthly, quarterly) meetings
Rotating Scrum Master of the S3, from within the leadership team
Rarely daily; typically weekly meetings
Discuss adjustments required to execute to the roadmap; cross-team &
cross-product line trade-offs
Focused towards to effectively managing the portfolio

Typical S4
Charter
Evangelizing & Guiding the Agile adoption – practices, evolution, meta-
retrospectives
Organizational focus:
Corporate leadership understanding & adaptation
Cross-functional understanding & adaptation (ALL functions)
Typically monthly meetings; Evolution Backlog / road-map
Rolling up retrospectives; enabling change
Training; Scrum Master & Product Owner focus groups
Focused towards to effectively managing the evolution of agility

Also deriving KPA’s and metrics for soft/hard ROI

Exercise
Gather in small groups; better that you are from the same
company/organization OR those with similar characteristics
Using the Scrum of Scrums notion and Scrum as your framework,
please define:
1) A hypothetical SoS hierarchy for your organization

2) Detail the format (timing, attendees, focus, etc.) for each level—sort
of the charter
3) Detail some of the “conversations” that might occur at each level
1) Status or state; progress & plans
2) Impediments
3) Improvement
4) Identify your primary impediment's to implementing this SoS
What Part do Tools Play
in the Enterprise? Point #1 – Agile
Methodologies are not tools
centric
Enterprise Drivers
Point #2 – Enterprise Agility –
Compliance / artifacts needs “something” to help
Sheer team size coordinate larger scale teams
Distributed teams
Project Management / PMO Point #3 – So,
Budgeting Use tools that are as simple
Project complexity as possible
Communications And, that don’t get in the
Status, metrics, and reporting way of your face-to-face
collaboration
Go out of your way to
reinforce collaboration

Agile Tools
Low Fidelity Medium Fidelity

3x5 Cards, Post-it Notes— Jira or other issue/defect tracking
planning on walls Advanced wiki functionality
Web cameras, conference SharePoint
phones, skype PlanningPoker, CardMeeting,
Webex, GotoMeeting, Wiki ScrumWorks, TargetProcess
Whiteboards, Corkboards
Swim lane management
Team rooms
High Fidelity
Rally
Continuous Integration VersionOne
(CruiseControl), Automated Mingle
testing – FitNesse, xUnit Microsoft Team Foundation
Server (TFS)

Aspects of Agile Scaling
Jeff Sutherland – Parallel Scrum
Sutherland is leading the way toward modifying Scrum towards
greater efficiency
Scrum levels – A, B, and C
Sutherland used Type C Scrum for 3-4 years at PatientKeeper

Sort of the “Nirvana” Scrum state, anyone else at Type C?
The key point is the organizational change dynamics

Sprint transition time compression
Forward thinking towards staging & delivery
Parallel --- Everything!
Organization-wide change implications – including Testing & Quality

Scrum Evolution
Parallel Pipelining
Type A Scrum
Isolated cycles of work
Type B Scrum
Overlapping Iterations
Backlog continuously
refreshed
Reduced staging times
Type C Scrum
All at once, multiple –
simultaneous releases
Organization of a Meta-
Scrum

Evolving Role of the Product Owner

Product Owners
Their Evolving Role
Within each Scrum, the PO is typically narrowly focused on crafting
the Backlog, engaging in progress, and reviewing sprint results
However, as Scrum scales, the PO team needs to become more

focused on:
Product Line Evolution – Meta Backlog and coordination across Sprint
teams, strategy development & execution, resource load-balancing, and
budgeting
Cross-Team Planning – SoS coordination, linked goals and backlog
work, delivery integration, and staged (forward-thinking) planning
Delivery Dynamics – timing, marketing, packaging, interrelationships,
customer feedback, and achieving production quality
All of course with the teams delivering the product

Product Owners
Guiding Testing
The PO also needs to have a Quality & Testing perspective that
within each Sprint focuses on –
Developing specific multifaceted quality release goals
Working with the team (Testers) to develop acceptance tests
Working with the team to ensure daily convergence towards goals
Across the SoS:

Developing Product & Portfolio quality meta goals that cross all Sprints
Integrating deliverables and qualifying the overall Product via planned
Integration & Stabilization Sprints
Balancing automation vs. manual testing capacity and investment
Focusing teams towards Product release points

Product Owners
Coordination Workflow
Stories Scrum of Scrum of Scrums –
Development
Workflow
The PO organization must
coordinate
Features Feature timing & workflow
Product Quality & integration workflow
Integration Product maturation and
Applets release readiness
Production deployment
Testing &
Applications Evolving In conjunction with technology and
Maturation project leadership
While often interfacing to operations and

Products customer facing organizations
Product Families

Product Owner
Planning Levels in Large-Scale Agile Projects
Product Vision Yearly by Product Owner (s)
Semi-Annually by the Product Owner (s)
Product Roadmap
Release Plans Quarterly by the Product Owner & Team (s)

Small
To
Large Monthly or bi-weekly by the Team (s)
Iteration Plans
Daily Plans Daily by the team members

Release Train Management
Iterative model with a release
target
Product centric
Focused on a production
push/release Notion of a “Hardening Sprint”
Focused more on Integration &
Synchronized Sprints across Regression testing
teams Assumption that it’s mostly
Some teams are un- automated
synchronized, but leads to less Environment promotion
efficient cross-team (product)
interactions Define a final Hardening Sprint
where the product is readied
Continuous Integration is the for release
glue Documentation, Support,
Including automated unit and Compliance, UAT, Training
feature tests; partial regression

“Motivational” Driving Forces
Vision
Schedule for delivery
Feature set(s)
Goals for each team
Themes
Which teams will be working on what components
Packages of User Stories
Prioritized by business value & need
End-to-End Use Cases

Integration focused execution

“Internal” Driving Forces
Customer’s ability to “accept” the release timing
Value being delivered within the release – purely scope
Hardening Sprint (production readiness “reality”)

Time, Complexity & Size
Levels of Test Automation
Compliance & Industry
Internal team readiness

Customer support
Sales & Marketing readiness
Overall documentation & training

Meta-Backlog – to – Backlog Translation
--- ---
Sprint
--- ---
--- --- ---
--- ---
--- ---
--- --- ---
--- ---
--- --- ---
Enterprise --- Sprint
--- --- ---
Architecture ---
--- --- ---
Product line ---
--- --- ---
Features sets and ---
--- --- ---
Themes --- Sprint
--- --- ---
Targeted release ---
--- --- ---
points ---
Integration & --- --- ---
testing --- --- --- Sprint
Documentation, --- ---
support training, ---
OPS readiness • Meta Backlogs are typically focused at multiple teams, working on a
Regulatory steps unified set of work, targeted for release
---
--- • It can also coordinate across separate product dependencies in a
--- shared codebase
--- • Or simply manage the shared codebase
---

The Agile Release Train
Synchronized
Internal External
Release Release
Team 1 Iterate Iterate Harden Iterate Iterate Iterate

Continuous Integration
Docs,
Team 2 Iterate Iterate Harden Iterate Iterate Iterate Training,
Continuous Integration X-team Support,
Harden UAT,
Team 3 Iterate Iterate Harden Iterate Iterate Iterate Comp.
Team 4 Iterate Iterate Harden Iterate Iterate Iterate

… Continuous Integration
Team n

The Agile Release Train
Example: eCommerce / SaaS Model
10 days 10 days 5 + 2 days
External
Release
Team 1 Iterate Iterate Harden

Team 2 X-team Rinse

Iterate Iterate Harden Harden
&
Team 3 Docs, Repeat
Iterate Iterate Harden
Training
Team 4
… Team 8 Iterate Iterate Harden
Environment
Dev + QA Dev + QA QA -> Staging Production
Evolution
Release Goal Setting
A Key for Coordination
As you scale, each planning level should create criteria (Sprint
Goals) that are –
Interrelated and cohesive
Focused towards the end product release and not simply on each teams
deliverables
Identify dependencies and overall workflow
The traditional notion of Chartering also applies at the higher levels,

with Charters defined as:
Goals, Objectives & Scope
Clearly measurable view to “Done” – Release Criteria
Multi-faceted view towards quality (defects, coverage, non-functional
requirements)
Allowing for team scope trade-offs

Establishing Goals & Criteria

Why it’s Crucial?
Agile teams are essentially self-directed, so plans don’t drive

behavior or success…
People do and Goals drive the Team!
The teams then swarm around the goals, using their creativity and
teamwork to figure out:
What’s most important

How to achieve it
Always looking for simple & creative—20% solutions

Lean Principles
Most important aspects first

KISS principle; no Gold-plating
Small deliverables; worked on serially
Deliver End-to-End behavior
In thin ‘Slices’
UI to Backend DB
Driven to done, into inventory as completed components, as soon as
possible!
Avoiding 90% done syndrome
Integration collaboration
Rarely re-visited; mindset is to ruthlessly minimize rework

Notions of Done-Ness
Need to define “Done” from team members perspectives
If you’re a developer, what does “I’m done with that story” mean?
Code complete
Code reviewed (paired)
Checked in – build successful
Unit tests developed – passed
Integration
QA collaboration
Run by the Product Owner
Every type of task should have a definition of Done-Ness! How else

could you estimate the work?
Story Acceptance
Each User Story should have acceptance criteria as part of the card
They should focus on the verifiable behavior, core business logic,

key requirements for the story
Typically, they are crafted by the Product Owner

Leveraging skills of Business Analysts and Testers
Story acceptance tests are normally automated and run as part of

feature acceptance AND regression
FIT & FitNesse are among the Open Source tools that enable this

User Story
Examples
As a dog owner, I want to sign-up

for a kennel reservation over
Christmas so that I get a
confirmed spot
Verify individual as a registered pet owner

Verify that preferred members get 15% discount on basic service
Verify that preferred members get 25% discount on extended services
and reservation priority over other members
Verify that past Christmas customers get reservation priority
Verify that declines get email with discount coupon for future services

Iteration / Goal Acceptance
Each Scrum Sprint has a Product Owner determined goal
Usually sprint success is not determined by the exact number of

completed stories or tasks
Instead, what most important is meeting the spirit of the goal
Deliver a 6 minute demonstration of the software that

demonstrates our most compelling value features and
achieves venture capital investment interest

Release Criteria
Goals and objectives for the entire project release
Usually they are multi-faceted, defining a broad set of conditions

Required artifacts & levels of detail
Testing activities or coverage levels
Quality or allowed defect levels
Results or performance metrics achievement levels
Collaboration with other groups – dependency management
Compliance levels
That IF MET would mean the release could occur.

Levels of Criteria
Activity Criteria Example

Basic Team Done’ness criteria Pairing or pair inspections of code prior to check-in; or
Work Products development, execution and passing of unit tests.
Development of FitNesse based acceptance tests with the
User Story or Acceptance Tests customer AND their successful execution and passing.
Theme Level Developed toward individual stories and/or themes for sets
of stories.
Sprint or Done’ness criteria Defining a Sprint Goal that clarifies the feature
Iteration Level development and all external dependencies associcated with
a sprint.
Defining a broad set of conditions (artifacts, testing
Release Level Release criteria activities or coverage levels, results/metrics, collaboration
with other groups, meeting compliance levels, etc.) that IF
MET would mean the release could occur.

Exercise
In this section, we discussed the various levels of criteria that are
important within agile teams.
1) Amongst your team, discuss how you’ve established goals & criteria
in your Agile (or non-Agile) teams
1) What levels are the most important?
2) What levels don’t matter as much?
2) What part does the team play in definition? Should play?
3) Does defining what “done” means really matter? How?
4) If you had only one to pick, which would it be? And why?

Scrum-in-Testing

Process Overview
Source: Adapted from Agile
Software Development with
The Agile intent is to Scrum by Ken Schwaber and
Mike Beedle.
perform all testing within
the iteration – usually
via automation.
Unit & Acceptance are
the typical focus, but 24 hours
Daily Scrum
what about other forms Meeting
of testing?
Legacy regression,
interoperability across
sprints, performance, Backlog tasks 30 days
usability, etc. Thus the expanded
rub! by team
Sprint Backlog
Potentially Shippable
Product Backlog Product Increment
As prioritized by Product Owner

Inherently Narrow Focus
of Agile Testing
Typical Agile Team What’s Missing for Larger Scale
Testing Focus Organizations?
Unit Testing
Integration Testing
Automated Builds –
Smoke Testing Functional Testing
Focused – Customer System Testing
Acceptance Testing Regression Testing
Performance Testing
Test Driven Development Load Testing
(TDD)
Very limited – Integration
& Regression Testing Scenario Based Testing
Focused Towards User Acceptance Testing (UAT)
Automation Usability Testing, Other “ility” Testing
Limited Exploratory Exploratory Testing
Testing
Large-scale Automation

Inherently Narrow Focus
of Agile Testing
Early as possible testing – unit
level TDD
Customer engaged in defining Think of Lean principles as an
acceptance criteria & tests underlying driver…
Early defect prevention,
detection & early repair Just Enough, Just in Time
High degree of automation – Deliver as fast as possible
“everything” runs within the Decide as late as possible
iteration Team integrity &
Quality as a team professionalism
responsibility Holistic, built-in quality
Ongoing refactoring as
required

Scrum-in-Testing
Transitional Drivers
High levels of traditional testing
Manual regression burden; Legacy systems; Habits across the business
stakeholder team; External pressures or expectations
PO requires this as part of the Backlog
Early Scrum implementations

Development teams struggled with gaining testing traction – not meeting
Agile code quality goals or core practices
Testing teams falling into traditional behaviors – artifact based,
gatekeeper mentality, and low adaptability or flexibility
Insufficient testing resources to staff Sprints and post Development
Sprint testing requirements (thinking that Agile teams inherently need
less testers)

Scrum-in-Testing
Coordination Drivers
Integration
Short term integration across (many) Sprinting Development teams
Integrating with external data providers
Integrating with 3’rd parties and outsourcers
Equipment limitations & production deployment / promotion models
The need to give an external team insights into deliverables for

including in future work
PO driven Portfolio & Product road-maps; PMO oriented planning
Regulatory or compliance requirements (traceability and/or artifact

evidence)
Scrum-in-Testing
Strategies
Extending testing activity within the Sprint to include as much
coverage as possible:
Of course, unit & acceptance for the current iteration features
Re-running existing unit & acceptance tests
Maintaining existing automation
Running partial integration & partial regression testing as possible
Cross-team collaboration
Which usually requires a different staffing mix for each Sprint
Or extending the iterative model to accommodate testing via –

Stabilization Sprints
Skewed Development & Testing focused Sprints

Scrum-in-Testing
Stablilzation Sprint(s) – focused on integrating development
release forward towards production release Testing Activities:
30 day
1. Full regression
Dev. 2. Overall integration
Sprint(s) 3. Performance
4. Usability
Variable length 5. Bug fixing
6. Production –
Sprint(s)
promotion steps
Production
30 day
Product Release
Dev.
Sprint(s)
Product Owner defined Product Backlogs

And “coordinates” between development sprints

Scrum-in-Testing
This is a modified 1. Pure Development focused Sprints –
version of the delivering features to PO to…
Scrum model
where Sprints 2. Early Integration focused Sprints –
evolve from ►►► coordinating a building product story and
shaking our interoperability dependencies to…
3. Pure Testing focused Sprints – performing

more traditional testing activity leading towards
Note: iteration production release.
resource mix
changes as you
move from 4. Finally and potentially Testing Infrastructural
development focused Sprints – automation maintenance,
towards next iteration readiness, and customer / usability
stabilization collaboration

Scrum-in-Testing
Stabilization Sprint – Sample Workflow
Early transition to 2 Stabilization Sprints

4 Development Sprints Stabilization Sprints
Strongly connected to
Normal team composition 50/50 Development + development, led by
Testing focus strong testing team
There is a resource transition

cycle from full Development
Sprints forward to full
Next development
Stabilization Sprints. Sprint beginnings –
The “Art” is in effective Iteration #0, gaining
collaboration, resource sharing traction, Sprint #1
& communication – forward & Repeat…
backward
Scrum-in-Testing
Stabilization Sprint – Content Pressure Release
Think of Stabilization Sprint(s) as a feature content pressure release
mechanism. As your content increases, so does its integration risk.
You’ll want to use them –
When you have many Development Sprints running in parallel
As an interim integration mechanism
When Stabilization Sprint threads run in parallel it creates the need

for S2 & S3 oriented interactions
The model typically is used for larger numbers of Development

Sprints contributing to a large-scale enterprise product
Duration and focus can vary from one Stabilization Sprint to the next
Although consistency matters

Scrum-in-Testing
Skewed Testing Sprints
Skewed Testing Sprint(s) – focused on providing more formal
testing feedback by virtually running development & testing in
parallel / skewed Sprints Testing Activities:
1. Partial regression
2. Limited integration
30 day 3. Early performance
(n) day Test 4. Bug fixing
Dev. 5. QA – promotion
Sprint(s) Sprint(s)
steps
Interim or Integration
Product Releases
Product Owner & test team members coordinate
bug feedback into current Development Sprint
Backlog and Product Backlog

Scrum-in-Testing
This model balances some traditional testing post Development
Sprint against bogging down the sprint w/too broad a level of testing
Usually the testing is focused towards traditional regression and

integration testing
May also be used for performance, compliance and other non-functional
testing activity
The model typically is used for domains with an existing large scale
legacy testing burden (or requiring larger scale testing practices)
In this case, the skew allows the development activity to progress while
testing is performed
Sometimes this is viewed as “Waterfall” testing within Scrum; but
necessary if you can’t perform ALL testing within the iteration
Scrum-in-Testing
Advantage of handling defects and integration issues close to the
originating Sprint
Can reserve Sprint Backlog time so that changes can be incorporated
immediately in the “next” Development Sprint
Testing Sprints are usually staffed solely with testers
In later phases, the testing can turn into more of a Stabilization

sprint effort – so a morphing of the two strategies
Over time as your Agile experience grows, you’ll want to narrow the
skew – perhaps with overlapping iterations

Testing-in-Scrum
Challenges

Testing-in-Scrum
Typical Challenges – Alignment
It’s important to try and align Scrum iteration tempo across your
SoSoS Sprints, putting pressure on your:
Cross-sprint Meta Backlog management & planning

Sprint Review results & feedback
Testing Sprint alignments
Dependency management
Lab support scheduling & deployment phasing
3’rd party integrations

Testing-in-Scrum
Typical Challenges – Resource Balancing
Resource Balancing
Between traditional, development focused Sprints…
Including people, equipment, and simply focus
Falling Behind
If you skew too heavily towards the traditional, testing stabilization
Sprints, you’ll lose connection to the next iteration
Falling Forward
If you skew too heavily towards the development Sprints, you’ll lose the
more formal testing & stabilization battle
More often – teams fall behind when they should be falling

forward…
Watch out for Waterfall Testing in an Agile World

Testing-in-Scrum
Typical Challenges – Skill-sets
Agile skill-sets and collaborative expectations are WAY different!
Do
Define agile testing behavior guideline (role & responsibilities)
Encourage pairing and strong collaboration
Assess your technical capabilities and begin to aggressively fill in any
gaps:
Technical domain understanding and direct programming experience
Open source automation tools experience
Customer collaborative and UAT experience
Establish guidelines for balancing between Agility & corporate quality
and governance expectations
Don’t underestimate the impact that Agility will have towards your
traditional teams’ capabilities, approaches and capacity for change
Testing-in-Scrum
Typical Challenges – Quality Influence
Working with the Product Owner (Customer)
Becoming a collaborative partner; defining & automating acceptance
tests
Actively representing the Voice of the Customer (VOC)
It’s important to setup clear & broad release criteria for each Sprint
and the overall Release
Feature goals; usability goals; acceptance confirmation
Quality criteria (defects, coverage, types of testing)
Process artifact goals (for example: SOX or other compliance,
traceability)
Establishing release readiness (features, quality, interoperability) for

PO during Sprint Review (Pass/Fail – goals met?)

Testing-in-Scrum
Typical Challenges – Metrics & Visibility
As the number of Scrums grow with application size, feedback is
gathered more so at the SoS level via –
Cross Sprint burndowns and feature tracking
Feedback from the testing team in integration issues
Product Owner Sprint Review(s) experience
Product Roadmap progress – across all of the relevant Sprints
The Meta ScrumMasters & Meta Product Owners are actively

engaged in defining goals and measuring progress against them
Test teams can / should provide some traditional metrics focused

towards –
Defects, test coverage & traceability, workflow defect patterns, Sprint
release acceptance / goal attainment levels

Testing-in-Scrum
Typical Challenges – Defects
In pure Agile teams (small teams & projects, quality & testing
focused, automation centered) there is little need for traditional
defect tracking techniques
They test first by developing unit tests and continuously integrating
changes;
Establish automated acceptance tests and fixing bugs as they’re found
However, in evolving or large-scale Agile teams

There is a need for defect coordination across the various product(s) and
team(s);
Traditional triage, and targeting repairs to specific teams & iterations
Product Owner(s) and ScrumMaster(s) are involved in this coordination
and delegation
Testing teams are at the center of these efforts; guiding the teams
towards their Sprint & Product Release goals

Exercise
The context here is your personal experience with any aspect of
Agile Testing
1) Describe some of the biggest challenges you’ve faced.

2) Can you envision some of the iterative model changes working in
your domains? What adjustments are needed?
3) Often a challenge is legacy code bases. Describe yours and, more
importantly, how have you attacked it to improve your “agility”?
4) Often the test teams themselves struggle moving towards agility,
describe your experiences with this. How have you improved these
situations?

Agile Endgames

Triage
Even in an agile environment there are bugs that surface late in the
game
Always bugs are—

Immediately made visible to the Product Owner
Team explores impacts and repair options
3 options: Fix, Defer to Product Backlog, Ignore
Triage is based on the (pre-established) Release Criteria view for

User Stories, the Sprint, and the overall Release
Always without the traditional fanfare associated with traditional

bugs
Role of Continuous Integration
From the Release Train, you’ll remember that continuous Integration

is sort of the glue for the entire process
Realize that you may have to make adjustments to the purist CI view
that everything moves on the trunk
Short lived branches might be required to be efficient in handling content
that is WIP and not releasing
Architectural implications
Ability to release products or components separately
Ability to hold back features or components that aren’t done

CI – Information Radiator
ChannelAdvisor

Environment Management
Product Maturation
Establishing an Environment & Promotion model
Development -> QA -> Staging -> Production
When and where are transitions made
Establish clear criteria for each transition; with back-out capabilities
Supplement the model with automated scripting to replicate

environments
Can be particularly challenging in DB intensive environments – stored
procedures & structure changes
Strongly consider coupling this to your CI efforts

Code Freeze Dynamics still Apply!
Try and front-load major features and high priority work
Develop milestones for coalescing your code towards a freeze point

Enter your hardening Sprints with a specific freeze target
During hardening have a coding halt target
May need layered freezes

UI versus Back-end Database layers
Database deployment script development
Still need to triage bugs to see what ‘fits’…usually in daily release

Scrum of Scrums

Microsoft - Code Complete Model
Microsoft employs a “code complete” strategy as defined below –
Various project Feature Code Complete

milestones Visual Freeze
Complete
Release to Stabilization leading to a

manufacturing zero bug fix release
Typical Activities:
Pilots
Alpha &Beta testing
Time buffer
Iterative testing and rework

Schoor – Example of Repair Code Freeze
Bruce Schoor has an extension to the Microsoft “code complete”

model as defined below –
Code Complete UBF LBF RC

Milestone Unlimited Bug Fix Limited Bug Fix Release Candidate
Testing Focus Systematic Production & Acceptance Tests

Test Passes Regression Tests
Release Goals Reduce # of High Drive to Zero No More Repairs

Priority Defects Defects & Release!
UBF – LBF LBF - RC

Gate Gate

Software Testing
Whenever possible…always take the next build or next release

Newer code always trumps coverage ---until you within your code freeze
handling
Test relentlessly; ruthlessly

Exploratory testing
Automation runs
80:20 Risk-based approaches (AllPairs)
Developer testing is required in the Endgame

Insightfull whitebox testing; leverage your unit tests (allowing them to
‘count’ towards coverage)
Paired with testers; focus on risk-based

Deployment Readiness
Focused on:
Support readiness
Production environment readiness
Customer readiness
Training
Usually achieved including different skills in Hardening or Pre-

Release Sprints
Brings up the point of Sprint team composition. Does it change over the
life of the Agile – Release Train?
What are some examples?

Release Retrospective
Should be conducted for each Release!
Apply your traditional format & facilitative rules

I like the What Worked, Needs to Change, Want to Try model for
gathering thoughts
Much broader engagement

For example, in a SaaS model engage Operations, Support, Customer
Account Managers, etc. for overall effectiveness
And stakeholders…perceptions? Calls from customers?
Log them and make action plans visible. Our CI “manager” served
as our Release Coordinator to focus our attention on the entire
cycle.

Wrap-up
Traditional Scrum (and other Agile methods) struggle to operate in:

Non-green field
Legacy based or compliance focused
Enterprise level or large-scale team
environments.
Primary focus points for successfully Being Agile in these contexts

include
Adapting your Customers towards Agility
Adapting Agility towards your Environment in a pragmatic fashion
Focus on the basics of the Agile Manifesto and basics of Scrum

Questions?
Thank you!

Core Agile References
Augustine, Sanjiv, “Managing Agile Projects”, Addison Wesley, (2005)
Beck, Kent, “Extreme Programming Explained – Embrace Change”, Addison
Wesley, (2005)
Beck, Kent and Fowler, Martin, “Planning Extreme Programming”, Addison
Wesley, (2001)
Cockburn, Alistair, “Agile Software Development – The Cooperative Game, 2’nd
Edition”, Addison Wesley, (2006)
Cockburn, Alistair, “Crystal Clear – A Human-Powered Methodology for Small
Teams”, Addison Wesley, (2005)
Cohn, Mike, “User Stories Applied – For Agile Software Development”, Addison
Wesley, (2004)
Cohn, Mike, “Agile Estimating & Planning”, Addison Wesley, (2006)
Derby, Esther and Larsen, Diane, “Agile Retrospectives – Making Good Teams
Great”, Pragmatic Bookshelf, (2006)
Highsmith, Jim, “Agile Project Management”, Addison Wesley, (2004)
Larman, Craig, “Agile & Iterative Development – A Manager’s Guide”, Addison
Wesley, (2004)
Leffingwell, Dean, “Scaling Software Agility – Best Practices for Large
Enterprises”, Addison Wesley, (2007)
Manns, Mary Lynn and Rising, Linda, Fearless Change – Patterns for
Introducing New Ideas”, Addison Wesley, (2004)

Core Agile References
Poppendieck, Mary & Tom, “Lean Software Development – An Agile Toolkit”,
Addison Wesley, (2003)
Poppendieck, Mary & Tom, “Implementing Lean Software Development – From
Concept to Cash”, Addison Wesley, (2006)
Schwaber, Ken and Beedle, Mike, “Agile Software Development with Scrum”,
Prentice Hall Publishing, (2002)
Schwaber, Ken, “Agile Project Management with Scrum”, Microsoft Press,
(2004)
Schwaber, Ken, “The Enterprise and Scrum”, Microsoft Press, (2007)
Tabaka, Jean, “Collaboration Explained – Facilitation Skills for Software Project
Leaders”, Addison Wesley, (2006)

Core Agile Web References
Mike Cohn – Scrum of Scrums page/picture:
http://www.mountaingoatsoftware.com/scrum/scrumteam.php
Jeff Sutherland: http://jeffsutherland.com

Agile 2005 paper on Type A, B, C Scrums: http://www.agile2005.org/RP10.pdf
Agile 2005 presentation on Scrum II:
http://jeffsutherland.com/scrum/ScrumIIAgile2005.pdf#search=%22scrum%20of%20sc
rums%22
Jeff Sutherland Agile 2006 presentation on Distributed Scrum Teams:

www.scrumalliance.org/index.php/content/download/4836/49524/file/SutherlandDistributed
ScrumAgile2006_v4_28_Feb_2006.pdf
Annual Scrum Gatherings and Agile conferences – www.agile2008.org

Alternate perspective on XP - http://www.softwarereality.com/ExtremeProgramming.jsp
Firms using Scrum - http://scrumalliance.pbwiki.com/Firms-Using-Scrum?doneSave=1
Planning Poker cards - http://contrado.com.au/PP_Cards.pdf
The history of the term SCRUM dates back to a 1986 article by Takeuchi and Nonaka in
Harvard Business Review. In it they connected the Rugby term to an iterative model for
product development. It's worth a read to simply see the history and connection back to
Lean Manufacturing practices - http://apln-
richmond.pbwiki.com/f/New%20New%20Prod%20Devel%20Game.pdf

Contact Info
Related ST&P articles in June 2006 and 2007 issues,
www.stpmag.com
Software Endgames: Eliminating Defects, Controlling

Change, and the Countdown to On-Time Delivery
published by Dorset House in Spring 2005.
www.rgalen.com for order info, misc. related
Robert Galen presentations, and papers.
RGalen Consulting Group, L.L.C.
PO Box 865, Cary, NC 27512
919-272-0719
www.rgalen.com
bob@rgalen.com


Agile Endgames

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Agile Endgames

Uploaded by

Copyright:

Available Formats

Releasing Agile Products

President & Principal Consultant,

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

We’ll be using Scrum as the Agile reference framework for

Agile Project Management Framework

Copyright © 2009 RGalen Consulting Grp,

Backlog tasks 30 days

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

Larger scale implementations: larger & distributed teams

Copyright © 2009 RGalen Consulting Grp,

There are really 4 aspects to Scaling Agility within the Enterprise:

Organizational – larger teams, distributed teams, and outsourced teams

Copyright © 2009 RGalen Consulting Grp,

Industry lessons have lagged in larger scale Agile implementations

In the Enterprise, Scrum is leading the way in providing guidance

Larger scale implications have been ignored to-date – thus this

Copyright © 2009 RGalen Consulting Grp,

Scrum of Scrums (of Scrums)

Scrum of Scrums - (S2)

Scrum Team Level - (S1)

Copyright © 2009 RGalen Consulting Grp,

1200 developers (S2) - Scrum of Scrums

Copyright © 2009 RGalen Consulting Grp,

60-70 developers (S2) - Scrum of Scrums

Copyright © 2009 RGalen Consulting Grp,

Who guides the process, tools, & techniques for consistency? Or do

How do you integrate agile processes with traditional business and

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

Focused towards the next, integrated product release point

Copyright © 2009 RGalen Consulting Grp,

Focused towards to effectively managing the portfolio

Copyright © 2009 RGalen Consulting Grp,

Focused towards to effectively managing the evolution of agility

Copyright © 2009 RGalen Consulting Grp,

1) A hypothetical SoS hierarchy for your organization

Copyright © 2009 RGalen Consulting Grp,

Low Fidelity Medium Fidelity

Copyright © 2009 RGalen Consulting Grp,

Sutherland used Type C Scrum for 3-4 years at PatientKeeper

The key point is the organizational change dynamics

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

However, as Scrum scales, the PO team needs to become more

All of course with the teams delivering the product

Copyright © 2009 RGalen Consulting Grp,

Across the SoS:

Copyright © 2009 RGalen Consulting Grp,

While often interfacing to operations and

Copyright © 2009 RGalen Consulting Grp,

Product Vision Yearly by Product Owner (s)

Semi-Annually by the Product Owner (s)

Release Plans Quarterly by the Product Owner & Team (s)

Daily Plans Daily by the team members

Copyright © 2009 RGalen Consulting Grp,

Copyright © 2009 RGalen Consulting Grp,

End-to-End Use Cases

Copyright © 2009 RGalen Consulting Grp,

Value being delivered within the release – purely scope