W4

Agile Testing

5/7/2014 11:30:00 AM

The Three Pillars Approach to

Your Agile Test Strategy

Presented by:

Bob Galen
Velocity Partners

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Bob Galen
Velocity Partners

An agile methodologist, practitioner, and coach based in Cary, NC, Bob Galen helps guide
companies in their adoption of Scrum and other agile methodologies and practices. Bob is a
principal agile evangelist at Velocity Partners, a leading agile nearshore development partner;
president of RGCG; and frequent speaker on software development, project management,
software testing, and team leadership at conferences and professional groups. He is a Certified
Scrum Coach, Certified Scrum Product Owner, and an active member of the Agile and Scrum
Alliances. In 2013 Bob published Scrum Product Ownership–Balancing Value from the Inside
Out. Reach him at bob@rgalen.com.
 The Three Pillars Approach to
Your Agile Test Strategy

Bob Galen
President & Principal Consultant
RGCG, LLC
bob@rgalen.com

Introduction
Bob Galen

Independent Agile Coach (CSC) at RGCG, LLC

Principle Agile Evangelist at Velocity Partners

Somewhere ‘north’ of 30 years overall experience ☺

Wide variety of technical stacks and business domains

Developer first, then Project Management / Leadership, then
Testing

Senior/Executive software development leadership for 20 years

Practicing formal agility since 2000

XP, Lean, Scrum, and Kanban experience

From Cary, North Carolina

Connect w/ me via LinkedIn and Twitter @bobgalen

Bias Disclaimer:
Agile is THE BEST Methodology
for Software Development
However, NOT a Silver Bullet!

Copyright © 2014 RGCG, LLC 2

1
Copyright © 2014 RGCG, LLC 3

Agile Testing & Quality

Let’s explore your strategies?

What are your current strategies towards:
 Software testing?
 And towards quality?

How are they “linked” together?

How does it “fit” with your overall agile strategy?

Get together in “pairs” and chat about this for 5-10

minutes.

Then we’ll gather your resultsB

Copyright © 2014 RGCG, LLC 4

2
3-Pillars
Genesis

First of all, I’ve seen way too many teams who are just
“testing” in agile teams without a map or a plan for
improvement

I’ve also seen that Agile Adoption is mostly a

programmer/developer or technology play. They “drive”
and testing is “along for the ride”.
 Seatbelts please!

And where’s the focus on “Quality”?

 And the how or practices & tactics

Copyright © 2014 RGCG, LLC 5

3-Pillars
Genesis

I’ve learned that “Balance” is important

A sad tale of:

 Thousands of ATDD testing; Gherkin run amok
 All of them are working; continuously testing; increasing
“coverage’ and life is Good!

BUT
 These same teams couldn’t write a cohesive User Story to save
their life
 So, where were the Acceptance Tests coming from?

Copyright © 2014 RGCG, LLC 6

3
3 Pillars of Agile Quality
Development & Test
Automation
• Pyramid-based Strategy:
(Unit + Cucumber +
Selenium)
• Continuous Integration
• Attack technical
infrastructure in the Backlog
• Visual Feedback –
Dashboards
• Actively practice ATDD and
BDD
• Whole Team Ownership of “Quality”
• Building it ‘Right’; Building the ‘Right’ Thing
• Center of Excellence or Community of Practice
• Strategic balance across 3 Pillars; Assessment, Recalibration, and Continuous Improvement
Copyright © 2014 RGCG, LLC
Foundation of the 3 Pillars
• Whole Team Ownership of
“Quality”
• Building it ‘Right’; Building
the ‘Right’ Thing
• Healthy – Agile Centric
Metrics
• Center of Excellence or
Community of Practice
• Strategic balance across 3
Pillars; Assessment,
Recalibration, and
Continuous Improvement
Copyright © 2014 RGCG, LLC
Software Testing Cross-Functional Team
Practices
• Risk-based testing:
Functional & Non-Functional • Team-based Pairing
• Test planning @ Release & • Stop-the-Line Mindset
Sprint levels
• Code Reviews & Standards
• Exploratory Testing
• Active Done-Ness
• Standards – checklists,
templates, repositories • Aggressive Refactoring of
Technical Debt
• Balance across manual,
exploratory & automation • User Stories, “3 Amigo”
based Conversations
• Healthy – Agile Centric Metrics
7
• Whole team view includes building it right,
everyone tests,
• Focus on features/stories, confirmation,
conversation, and getting them staged
properly OVER testing
• 4-tier metrics: Quality, Value, Prediction, Team
• Agile strategies need light-handed “steering”;
establish a CoE (heavier weight) or a CoP
(lightweight)
• Consider finding an assessment framework
and then tying it to your strategy
measurement, recalibration, and continuous
improvement.
• Make the Foundation visible thru Information
Radiators and metrics
8
4
3 Pillars of Agile Quality

Development &
Test Automation A central part of agile adoption is focusing on CI, 3-
tiered Automation development, and Dashboards to
• Pyramid-based begin incrementally building coverage for faster
Strategy: (Unit + feedback on changes.
Cucumber + Selenium)
In the interim, Hardening or Stabilization Sprints and
• Continuous Integration having a risk-based Release Train concept help

• Attack technical It’s important that Test or QA not ‘own’ the tooling or
infrastructure in the all of the automation efforts. The strategy can come
Backlog from Test, but the tactical automation development is
best left to the team.
• Visual Feedback –
Dashboards Mature teams invest in automation as part of Done-
ness and continually on their backlogs
• Actively practice ATDD
and BDD

9
Copyright © 2014 RGCG, LLC

3 Pillars of Agile Quality

Exploratory Testing (Charter / Session based and
paired) can be an incredibly effective way to
Software Testing establish a whole-team, collaborative view towards
quality and testing. It also emerges new tests.
• Risk-based testing:
Functional & Non- Leverage ‘plans’ as a whole-team collaboration
Functional mechanismBand do plan.

• Test planning @ Do not measure testing or tester progress; instead,

Release & Sprint levels
• Exploratory Testing
• Standards – checklists,
templates, repositories
measure throughput, output, sprint outcomes, and
done-ness escapes at a team level.
You need a balanced test team; not everyone needs
to be able to program. But everyone needs to be
skilled testers.

• Balance across Agile testing is a Risk-Based play in every Sprint and

manual, exploratory & across a release sequence. Don’t forget your
automation techniques!

10
Copyright © 2014 RGCG, LLC

5
3 Pillars of Agile Quality

Cross-Functional One of the hardest areas to get ‘right’ culturally. It

Team Practices needs leadership alignment from Quality/Testing to
Product to Development and a consistent voice of
whole-team approaches.
• Team-based Pairing
This is where LEAN lives, where whole-team
• Stop-the-Line Mindset
collaboration happens, where professionalism and
craftsmanship are held dear.
• Code Reviews &
Standards
I like the view of testers becoming the VOC,
champions of quality, and consistent questioners of
• Active Done-Ness
what is being build. Are we solving the right
problemsBas simply as possible. Notions of Minimal
• Aggressive Refactoring
Viable Product / Feature help with focus.
of Technical Debt
And yes Virginia, there ARE standards, templates,
• User Stories – 3 Amigo
and a focus on consistency!
based Conversations

11
Copyright © 2014 RGCG, LLC

Software Testing
Strategies

It ALL starts with empowering testers AND creating a
Whole-Team view towards Quality

Critical Early Steps:

 Creating a sense of empowered Functional Team
 Applying Testing Standards across all teams
 Deploying Exploratory Testing across all teams
 Defining a core set of Agile KPI / metrics
 ACTIVE participants in Sprint Planning

Copyright © 2014 RGCG, LLC 12

6
Cross-Functional Team Practices
Strategies

Training
 Agile / Lean in general, Story writing, Acceptance, Unit testing,
etc.
 Teaming – for example: feedback or 5 Dysfunctions / Trust

Critical Early Steps:
 Coaches & Scrum Masters to reinforce: Pairing / Swarming; WIP
Limits across teams
 Define prescriptive and aggressive Done-Ness for ALL teams
 Implement coding standards & Crucible / code reviews across the
center (appropriate for technology stacks)
 Release Planning BEFORE allowing a team to start Sprint #1
 Backlogs have Bug + Refactoring + Automation targets (20%)?

Copyright © 2014 RGCG, LLC 13

Organizational Quality
Strategies
Continuously communicate your unified Vision

Your strategy must be aligned/shared across:

 Development, Quality/Testing, and Product

Keep working your strategy across the pillars
 Don’t get stuck with too narrow a focus (easy road)

Make your strategy visible (Information Radiators)
 Show progress (Ex: burn up of test automation coverageBacross tiers)

Visualize organizational impediments to your Agile Quality
strategies
 Attack them!

Quarterly read-outs on progress, plans and adjustments
 Listen to your teams; Celebrate successes!

Copyright © 2014 RGCG, LLC 14

7
Contact Info

Bob Galen
Principal Consultant,
RGalen Consulting Group, L.L.C.

Experience-driven agile focused training,

coaching & consulting

Cell: (919) 272-0719

bob@rgalen.com www.rgalen.com
bgalen@velocitypartners.net www.velocitypartners.net

Blogs
Project Times - http://www.projecttimes.com/robert-galen/
BA Times - http://www.batimes.com/robert-galen/

Podcast on all things ‘agile’ - http://www.meta-cast.com/

Copyright © 2014 RGCG, LLC 15

Additional Topics

Copyright © 2014 RGCG, LLC 16

8
Two Pillars of Lean ‘Thinking’

Respect for Continuous

People Improvement

 Customer, Employees,  Embrace change,

VendorsB challenge everything
 Develop your teams  Kaizen – small, incremental
change
 Trust & coach
 Kaikaku – larger scale,
 No wasteful work
fundamental

From http://www.leanprimer.com

Copyright © 2014 RGCG, LLC 17

Agile Testing Quadrants

Brian Marick; Lisa Crispin & Janet Gregory

Automated & Business Facing

Manual
Manual

Functional tests Exploratory testing

Story tests Scenarios
Examples Usability testing
Supporting the Team

Critique the Product

Prototypes UAT
Simulations Alpha / Beta
Q2 Q3

Q1 Q4

Unit tests Performance testing

Component tests Load testing
API tests Security testing
Non-functional requirements

Automation,
Automated &
Tools, and
Manual
Technology Facing Manual

Copyright © 2014 RGCG, LLC 18

9
Agile Test Automation Pyramid
Mike Cohn; Lisa Crispin & Janet Gregory
http://behaviordrivendevelopment.wikispaces.com/Testing

Copyright © 2014 RGCG, LLC 19

10 Tenets of Agile Testing Jean Tabaka,

Rally Software
1. The system always runs
Continuous Integration

2. Stop the line, vs. logging
Lean – fix it now!

defects

3. If it’s not tested, it’s not
Early feedback; Earned

“Done” Value

4. Testing comes first, not last
Collaborative testing, focus

on building in quality
5. Finding defects after
Development is “Done” is too
Early feedback; fix it now!
late

Copyright © 2014 RGCG, LLC 20

10
10 Tenets of Agile Testing Jean Tabaka,
Rally Software
6. “Development Complete” is
Whole Team complete view
meaningless – no “partial credit”

7. Use testing, not analysis, to
Executable requirements

explore requirements

8. Automation is “how” not a
Automate all testing;

“whether” or “when” feedback

9. Tests are your second most
Code is first; later is

detailed specification traditional specifications

10. Testers are Customer-
VOC; guide effective team

Developer liaisons collaboration; ask the right
questions
Copyright © 2014 RGCG, LLC 21

10 Commitments of Agile Testing Jean Tabaka,

Rally Software

1. We commit to not moving forward if a hole is found

through root cause analysis without first writing a test
2. We commit to not relying solely on just automated
testing or just manual testing
3. We commit to not sitting behind a QA wall (no
boundaries!)
4. We commit to not allowing a code complete without
test code harness complete
5. We commit to not waiting for a test phase but rather
working in smaller and smaller pieces, sooner and
sooner

Copyright © 2014 RGCG, LLC 22

11
10 Commitments of Agile Testing Jean Tabaka,
Rally Software

6. We commit to not testing one iteration after

development is “Done”
7. We commit to not allowing surprises to accumulate for
large end-to-end testing (“mock it now”)
8. We commit to not leaving the riskiest tests to the end
9. We commit to being an equal participant with the
customer and the developer in defining “Doneness”
10. We commit to not taking this oath lightly

Copyright © 2014 RGCG, LLC 23

12