Acceptance Test Driven Development 119718667842574 3

Acceptance Test Driven
Development
Naresh Jain
naresh@agilefaqs.com
copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.
1
Tutorial Schedule
• Part discussion, part exercise

• Starts light and fluffy
• You’ll get your hands dirty by the end
• Questions welcomed all the time
2
Welcome
3
Welcome
• Continuum
3
Welcome
• Continuum
• Hopes and Concerns
3
Getting Started Exercise
Warmup Scenarios
4
Warmup Scenarios
• Going out for Movie (THX sound and Digital

projection)
4
Warmup Scenarios

projection)
• Going out for meal (one veg.)
4
Warmup Scenarios

projection)
• Going shopping ($50)
4
Warmup Scenarios

projection)
• Going shopping ($50)
• You are a party of 5.
4
What are
Acceptance Tests?
5
What are Acceptance Tests?
Criteria for Completion
• Written by the “Customer”

• Given to developers
6
Communication
• Forces customers, testers, and developers to
work together
• creation
• implementation
• execution
• maintenance
7
Feedback
• A source of data
• A tool for project management
8
Data From Acceptance Tests
Total ATs Failing ATs Passing ATs
90
72
54
36
18
0
1 2 3 4 5 6 7 8 9 10
9
Acceptance Tests:
A Critical Piece of Agile
10
Acceptance Tests: A Critical Piece of Agile
Traditional Approach
1 May 1 Jul 1 Sep 1 Nov
Analysis
Design
Implementation
ERD
DFD
DD
ST
11
The Agile Aproach

May 1 Jul 1 Sep 1 Nov 1
Analysis
Test
Implementation
Design
12
The Agile Approach
13
The Agile Approach

Acceptance
Criteria
13
The Agile Approach

Acceptance
Criteria
13
The Agile Approach

Iteration
Acceptance
Criteria
13
The Agile Approach

Iteration
Automated
Acceptance Acceptance
Criteria Tests
13
The Agile Approach

Iteration
Automated
Criteria Tests
13
The Agile Approach

Iteration
Automated
Criteria Tests
Automated
Acceptance
Tests
13
The Agile Approach

Iteration
Automated
Criteria Tests
Automated
Acceptance
Tests
13
The Agile Approach

Iteration
Automated
Criteria Tests
Automated Acceptance
Acceptance Criteria
Tests
13
Criteria for DONE
• Every story must have at least one

Acceptance Test
• A story is not DONE until it passes it’s
Acceptance Tests
14
Manual Acceptance Tests
15
Manual
Acceptance Tests
15
Manual
Acceptance Tests
15
Acceptance Tests Are
Automated
16
The Button
• How often would
you press it?
• When would you
press it?
• Who would press
it?
• Testers, Developers,
Managers, Customers,
Spectators, etc.
17
Automated Tests Require:
• High level, yet, precise language

• Unambiguous detail
• “Examples”
• Interaction with the system
18
19
Criteria for DONE
19
Criteria for DONE
+
19
Criteria for DONE
+ Automated
19
Criteria for DONE
+ Automated
Executable Specification
19
Executable Specification
• A new paradigm for testing

• Puts quality first
• Removes ambiguity from requirements
20
Who Writes Acceptance
Tests?
21
Who Writes Acceptance Tests?
The Customer
• Yeah right! Then who?
• The Customer Role
• Stake holder
• Business Analyst
• Quality Assurance
• Product Owner
• Developercopyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved.
22
Tests Get Technical
• The “Customer” may need technical help to

write tests
• Developers and QAs are technical
• Pair test authoring
23
Business Rules Get Fuzzy
• Sometimes developers need help

understanding tests
• Customers know business rules
• Pair test implementation
24
Business Decisions are

Hard To Make
• Sometimes, the “customer” will be

challenged to make decisions
• The Man ($$$) can help
• Do you see a pattern of communication?
25
Exercise #1
26
Exercise #1
The Login Test

• Write a test plan, in plain text, for
the business rules of logging in.
• Web application
• User credentials are stored in
relational database
• Successful login redirects to
“Welcome” page
27
Writing Good
Acceptance Tests
28
Writing Good Acceptance Tests
Login Test Possibilities
1. Direct browser to URL for login page
29
29

1. Enter the username ‘wallace’
29

29

Build a Testable Environment First
29
BOC
Build
Operate
Check
30
1. Add some users to the system
31
31

3. Enter a value into the username field
31

31

Be Specific
31
Tests are Examples
• Use concrete examples

• Specify concrete behavior
• No ambiguity allowed
32

1. Insert into User table values (’wallace’,
‘ilikecheeze’)
33

‘ilikecheeze’)
33

‘ilikecheeze’)
2. Open a browser to the URL http://
localhost/myapp
33

‘ilikecheeze’)
localhost/myapp
33

‘ilikecheeze’)
localhost/myapp
Avoid Implementation Details
33
Avoid Implementation Details
Tests UI
System
Database
34
Login Test: Possible Solution

• Add user to system: (’wallace’, ‘ilikecheeze’)
• Process login with username ‘wallace’ and
password ‘blah’
• Check login failed
• Process login with username ‘wallace’ and
password ‘ilikecheeze’
• Check login succeeded
35
Tools
36
Tools
Commercial Tools
• WinRunner TestPartner EggPlant
• Silk QTP TestComplete
• RFT Squish WindowTester
Are not suitable for Acceptance Testing in

an Agile environment
37
Tools
Open Source Options

• FIT Sahi Frankenstein
• FitNesse Watir SharpRobo
• Selenium Abbot WET
Among the few tools that support Test

Driven Development
38
Tools
FIT
• Framework for Integrated Tests

• Created by Ward Cunningham
• Open Source
• The most accepted solution for agile
acceptance testing
39
Tools
FitNesse
• Environment build around FIT

• Makes everything easier
• Created by Object Mentor, Inc.
• Open Source
40
FIT FitNesse
- Stand alone web server

- Is a wiki
- Tests written in HTML
- Tests written in wiki text
- Tests are executed on the command line
- Tests are executed from within the wiki
- Tables are executed
- Translates tests into HTML
- Non-table markup is ignored
- Uses FIT to execute tests
- Tables map to Fixtures
- Supports test suites
- Fixtures are code that is aware of the
- Supports variables in tests
system
- Supports test refactoring
- Supplies foundational Fixtures
- Written in Java
- Implementations ported to many
- Supports FIT implementations in any
languages
language
41
Thinking in Tables
42
Thinking in Tables
Only Tables Execute
Ignored
Executed
43
Thinking in Tables
Foundational Table Structure

Name of Fixture
Interaction with Application
• Table structure depends on type of Fixture
44
Thinking in Tables
3 Foundation Fixtures
• Column Fixture
• Row Fixture
• Action Fixture
45
Thinking in Tables
Column Fixture
package eg;
// Copyright (c) 2002 Cunningham & Cunningham
// Released under the terms of the GNU Genera
Public …
import fit.ColumnFixture;
public class Division extends ColumnFixture {

public float numerator;
public float denominator;
public float quotient() {
return numerator / denominator;
}
}
46
Thinking in Tables
Row Fixture
Analogous to comparing
against rows in a
database table
package fitnesse.fixtures;
import fit.RowFixture;
public class EmployeePayRecord {
public int id;
public class EmployeePayRecordsRowFixture extends private double salary;
RowFixture { public EmployeePayRecord(int id,
public Object[] query() throws Exception { double salary){
EmployeePayRecord[] records = new this.id = id;
EmployeePayRecord[2]; this.salary = salary;
records[0] = new EmployeePayRecord(1, 1000); }
records[1] = new EmployeePayRecord(2,2000); public double pay() {
return records; return salary;
} }
public Class getTargetClass() { }
return EmployeePayRecord.class;
}
}
47
Thinking in Tables
Action Fixture
• Think GUI window Counter Window
Counter:
public class CountFixture extends Fixture {
private int counter = 0; Counter: 6
public void count() { Count

counter++;
}
public int counter() {

return counter;
}
}
48
Exercise #2
49
Exercise #2
Login With Tables

• Write the Login test using tables
• Keep in mind what we learned from the
previous exercise
• BOC
• Examples
• Avoid Implementation Details
50
Exercise #2
Possible Solution
51
FitLibrary
• Extension to FIT
• Written by Rick Mugridge
• Adds some handy Fixtures
52
FitLibrary
FitLibrary Fixtures
• ArrayFixture for ordered lists
• SetFixture for unordered lists
• SetUpFixture
• Supports
• Graphics
• Tree structures
• Nested Tables
53
FitLibrary
DoFixture
• Broken tables Text
• Highly readable
• Flexibility
54
Wiki
55
Wiki
What is it?
• A collaborative web site
• Editable by any
• Created by Ward Cunningham
• Every project should have one
• http://c2.com/wiki
• http://en.wikipedia.com
56
Wiki
Creating Tests
• Use Wiki syntax to create a page with test tables
• Label the page as a Test Page
• Use a page name of the form Test…
• Turn on the Test property
• Make sure your Fixtures are in the classpath
• Use !path widget
• Mechanics
• !path values are concatenated
• Java command to start FitServer is executed
• Testable HTML is passed to FitServer
• FitServer runs the tests
• Results are passed back to FitNesse
57
Wiki
Creating Suites
There are 2 ways to make Suites
• Set the Suite property
• Create a page with the Suite property
• Created test pages inside this page
• When the suite is executed, all child test pages will
be included in the suite execution
• Use the !see widget
• !see <name of test page>
• All “included” tests pages will be included in the
suite execution
Run a Suite by clicking the Suite button
58
Hands-on Session
• Conference Proposal Submission Portal
• Some sample Stories
• Should be able to submit new proposal
• Should be able to list all submitted proposal
• Submitting proposal with same title should display appropriate error
message
• Should be able to delete submitted proposal based on the title
• Should be able to delete submitted proposal based on the title
• Should be able to search proposals by title
• Should be able to search proposals by ID
• Should be able to find all proposal by an author's name
59
Break
http://www.fitnesse.org
60
Patterns
61
Patterns
Organizing Tests
• Allowing customers to add new tests

without breaking the build
62
Patterns
Version Control
• Keeping the acceptance test in version

control with the code.
63
Patterns
Cross-Functional Pairing
• Using FitNesse based acceptance tests for

collaboration between cross-functional team
members.
64
Patterns
ATDD
• Acceptance Test Driven Development
65
Patterns
CSTT
• Cleanup, Setup, Test, Teardown
66
Patterns
Independent Tests
• Tests shouldn’t depend on each other.

• Tests leave the system in the same state it
started in.
67
Patterns
Dynamic Stubbing
• Avoiding complications of external systems.
68
Non-Production
Patterns
Setup/Teardown
• Using non-production light weigh code for

setup and teardown.
• Helps test only what you want to test.
69
Patterns
Suite Levels
• Creating different levels of suites depending

on the depth/level of feedback desired.
• Smoke, Current Iteration/Sprint, Regression
70
Patterns
DRY
• Using !include to avoid repeating yourself.
71
Patterns
Make it Real
• Write ATs as close as possible to the real

environment.
72
Patterns
Fixture Evolution
• Allow Fixture implementation to evolve over

time.
• Treat fixtures as first class citizens.
73
Patterns
At Least One Test/Story
• Every story should have at least one

acceptance test
• Avoid long/multipurpose tests.
74
Anti-Patterns
75
Anti-Patterns
Developer ATs
• Developers writing acceptance tests by

themselves, for themselves.
76
Anti-Patterns
Unit Testing
• Don’t write ATs at the unit testing level

• Unit tests are implementation specific
• ATs are NOT implementation specific
77
Anti-Patterns
QA Testing Tool
• Hard to write tests up front.

• Perhaps only on large projects.
78
Anti-Patterns
Silver Bullet
• Trying to use FitNesse for all types of

Acceptance Tests
• UI testing
• XML testing
79
Anti-Patterns
Test After
• Writing tests after the code is already

written.
80
Anti-Patterns
Hidden Test Data
• Hiding test data in the fixtures.
81
Implementation
Anti-Patterns
Dependant ATs
• Making test pages (tables) dependent on

implementation details and data structures.
82
Anti-Patterns
Logging in Your Fixtures
• Putting log statements or print statements in

the fixture code.
• Fixtures are probably too complicated.
83
The End
http://www.fitnesse.org
84

Acceptance Test Driven Development 119718667842574 3

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Acceptance Test Driven Development 119718667842574 3

Uploaded by

Copyright:

Available Formats

Acceptance Test Driven

• Part discussion, part exercise

• Going out for Movie (THX sound and Digital

• Going out for Movie (THX sound and Digital

• Going out for Movie (THX sound and Digital

• Going out for Movie (THX sound and Digital

Criteria for Completion

• Written by the “Customer”

The Agile Aproach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

The Agile Approach

Criteria for DONE

• Every story must have at least one

Manual Acceptance Tests

Manual Acceptance Tests

Manual Acceptance Tests

Automated Tests Require:

• High level, yet, precise language

• A new paradigm for testing

Tests Get Technical

• The “Customer” may need technical help to

Business Rules Get Fuzzy

• Sometimes developers need help

Business Decisions are

• Sometimes, the “customer” will be

The Login Test

Login Test Possibilities

1. Direct browser to URL for login page

Login Test Possibilities

1. Direct browser to URL for login page

Login Test Possibilities

1. Direct browser to URL for login page

Login Test Possibilities

1. Direct browser to URL for login page

Login Test Possibilities

1. Direct browser to URL for login page

Build a Testable Environment First

Login Test Possibilities

1. Add some users to the system

Login Test Possibilities

1. Add some users to the system

Login Test Possibilities

1. Add some users to the system

Login Test Possibilities

1. Add some users to the system

Login Test Possibilities

1. Add some users to the system

Tests are Examples

• Use concrete examples

Login Test Possibilities

Login Test Possibilities

Login Test Possibilities

Login Test Possibilities

Login Test Possibilities

Avoid Implementation Details

Avoid Implementation Details