David Noble

dnoble@exist.com Sr. Solutions Architect, Exist Global

Testing in Java
San Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007

Goal
Motivate excellence in testing
with concepts, practices, and tools

2

Scope
requirements quality assurance

continuous integration

Testing concepts, practices, tools
profiling

debugging

static code analysis

3

Agenda
Test-Driven Development Terminology Tools Application Discussion

4

System Concept System Requirements System Design Detailed Design Implementation
T es ti n g
http://www.flickr.com/photos/h2oalchemist/342496829/

5

Demo
A Taste of TDD

Test-Driven Development
Add test Fail Make small change Pass Commit changes
See also: http://www.agiledata.org/essays/tdd.html
7

Pass

Test-Driven Development
Add test Fail Fix bug Pass Commit changes
8

Test-Driven Development
Refactor Code
Pass

Commit changes Add test
Fail

Make small change
Pass

Pass

Commit changes
9

Test-Driven Development
Small steps Frequent check-ins

10

Test-Driven Development
Deeper than “test first” Design for testability

11

Test-Driven Development
Unit tests as example code Interface before implementation
Function drives form

12

Test-Driven Development
Loose coupling Services with clear responsibilities Mock objects

13

Taste the soup!
14
http://flickr.com/photos/basak/227480302/

Intent
Requirements, Architecture, Design

Reality
Implementation, Deployment

15

Intent
Requirements, Architecture, Design

Reality
Implementation, Deployment

G

ap !

16

Intent
Requirements, Architecture, Design

Accountability Reality
Testing, QA Implementation, Deployment

17

Intent
Requirements, Architecture, Design

Accountability Reality
Testing, QA Implementation, Deployment

R C ea he li ck ty !

18

Terminology
QA – quality assurance
Do the right things Do things right Ensure quality standards

19

Terminology
Test – success criteria
Is it right? Is it done?

20

Terminology
Test plan Test case Test suite Test report Test coverage

21

Terminology
Unit test Functional test Integration test System test Acceptance test Regression test
22

Terminology
Non-functional testing Performance test Load test Soak test

23

Terminology
Continuous integration

24

Tools

Mock Objects Unit Tests Build IDE

jMock, EasyMock JUnit, TestNG Ant, Maven Eclipse, NetBeans

25

Tools

Web Apps JEE Servers Databases

Selenium, HttpUnit Cactus DBUnit

26

Tools

Acceptance Selenium, FitNesse, Stiq Performance JMeter, Eclipse TPTP Coverage Cobertura, Emma

27

Tools

Continuous Integration

Cruise Control, Continuum, Hudson

28

Tools
Continuous Integration Acceptance Performance Coverage Cruise Control, Continuum, Hudson Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma Selenium, HttpUnit Cactus DBUnit jMock, EasyMock JUnit, TestNG Ant, Maven Eclipse, NetBeans
29

Unit testing

Web Apps JEE Servers Databases Mock Objects Unit Tests Build IDE

Demo
Tools

Resources: SGV JUG
Web Site
http://sgvlug.net/mailman/listinfo/java-sig

Mailing List (Hosted by SGV Linux Users Group)
http://sgvlug.net/mailman/listinfo/java-sig java-sig@sgvlug.net

31

Resources: TDD
Articles
http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)

32

Resources: IDE & Build Tools
Eclipse IDE
http://www.eclipse.org/

NetBeans IDE
http://www.netbeans.org/

Ant
http://ant.apache.org/

Maven 2
http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM

Maven & Eclipse
http://m2eclipse.codehaus.org/

Maven & Netbean
http://maven.apache.org/guides/mini/guide-ide-netbeans/guide-ide-netbeans.html
33

Resources: Unit Testing
JUnit Framework
http://junit.org/

JUnit FAQ
http://junit.sourceforge.net/doc/faq/faq.htm

JUnit with Ant
http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/manual/OptionalTasks/junitreport.html

JUnit with Maven
http://maven.apache.org/plugins/maven-surefire-plugin/

TestNG Framework
http://testng.org/doc/

Jester – Mutation Testing
http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
34

Resources: Mock Objects
Mock Objects Blog
http://mockobjects.com/

EasyMock Library
http://easymock.org/

jMock Library
http://jmock.org/

35

Resources: Unit & Functional
XmlUnit Library
http://xmlunit.sourceforge.net/

DBUnit Library
http://dbunit.sourceforge.net/

HttpUnit Library
http://httpunit.sourceforge.net/

Cactus for JEE Server Code
http://jakarta.apache.org/cactus/

Cargo for JEE Server Code
http://cargo.codehaus.org/

36

Resources: Test Coverage
Cobertura
http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/

Cobertura with Ant
http://cobertura.sourceforge.net/anttaskreference.html

Cobertura with Maven
http://mojo.codehaus.org/cobertura-maven-plugin/usage.html

Emma
http://emma.sourceforge.net/

37

Resources: Perf. & Load
JUnitPerf
http://clarkware.com/software/JUnitPerf.html

JMeter
http://jakarta.apache.org/jmeter/

Eclipse TPTP (Test & Performance Tools Platform)
http://www.eclipse.org/tptp/

Netbeans Profiler
http://profiler.netbeans.org/

38

Resources: Acceptance Tests
Selenium for Web Apps
http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action

Automating Selenium (“Remote Control”)
http://www.openqa.org/selenium-rc/
http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration

Selenium & Maven
http://raibledesigns.com/rd/entry/selenium_plugin_for_maven

FitNesse
http://fitnesse.org/

Stiq
http://storytestiq.sourceforge.net/

39

Resources: Cont. Integration
Cruise Control
http://cruisecontrol.sourceforge.net/

Continuum
http://maven.apache.org/continuum/

Hudson
https://hudson.dev.java.net/

40

Resources: App used in Demo
Open Source Medical Record System
http://www.openmrs.org/

Prerequisites: SVN, MySQL 5, Tomcat 5
http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/

SVN Repository
http://svn.openmrs.org/openmrs/branches/alpha

Installation
Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
41

Thank you
To all who participated!

David Noble
dnoble@exist.com Sr. Solutions Architect, Exist Global

Testing in Java
San Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007

My background: Mostly C & Perl from 1990 – 1997 Mostly Java since then. Telemetry processing, mission ops, EIS @ JPL. BPM, workflow, assorted biz apps.

1

Goal
Motivate excellence in testing
with concepts, practices, and tools

2

Goal falls within the larger context of pursuing excellence in software development.

2

Scope
requirements quality assurance

continuous integration

Testing concepts, practices, tools
profiling

debugging

static code analysis

3

3

Agenda
Test-Driven Development Terminology Tools Application Discussion

4

4

System Concept System Requirements System Design Detailed Design Implementation
Te st in g
http://www.flickr.com/photos/h2oalchemist/342496829/

5

In traditional software development, testing is usually at the end of the schedule. If development runs late, the testing often suffers.

5

Demo
A Taste of TDD

6

Test-Driven Development
Add test Fail Make small change Pass Commit changes
See also: http://www.agiledata.org/essays/tdd.html
7

Pass

7

Test-Driven Development
Add test Fail Fix bug Pass Commit changes
8

8

Test-Driven Development
Refactor Code
Pass

Commit changes Add test
Fail

Make small change
Pass

Pass

Commit changes
9

9

Test-Driven Development
Small steps Frequent check-ins

10

10

Test-Driven Development
Deeper than “test first” Design for testability

11

11

Test-Driven Development
Unit tests as example code Interface before implementation
Function drives form

12

12

Test-Driven Development
Loose coupling Services with clear responsibilities Mock objects

13

13

Taste the soup!
14
http://flickr.com/photos/basak/227480302/

An old man calls the waiter over to his table. "What would you like, sir?" asks the waiter. "Taste the soup," he says. "Is the soup too cold?" asks the waiter. "Taste the soup," says the old man. "Is it too salty?" asks the waiter. "Taste the soup!" says the old man. "Is there a fly in it?" asks the waiter. "Taste the soup!" the old man insists. The waiter looks down: "There's no spoon!" "Aha!" exclaims the old man.

14

Intent
Requirements, Architecture, Design

Reality
Implementation, Deployment

15

15

Intent
Requirements, Architecture, Design

Reality
Implementation, Deployment

G

ap

!

16

16

Intent
Requirements, Architecture, Design

Accountability Reality
Testing, QA Implementation, Deployment

17

17

Intent
Requirements, Architecture, Design

Accountability Reality
Testing, QA

R Ch ea ec lity k!

Implementation, Deployment

18

18

Terminology
QA – quality assurance
Do the right things Do things right Ensure quality standards

19

Miriam-Webster: “a program for the systematic monitoring and evaluation of the various aspects of a project, service, or facility to ensure that standards of quality are being met”

19

Terminology
Test – success criteria
Is it right? Is it done?

20

20

Terminology
Test plan Test case Test suite Test report Test coverage

21

Test plan: document Test case: implementation of specific test. Test suite: collection of test cases. Test report: results of running the tests. Test coverage: description of how much code (and which code) was exercised by tests.

21

Terminology
Unit test Functional test Integration test System test Acceptance test Regression test
22

Unit test: typically at the granularity of a class. Functional test: a feature or use case. Integration test: multiple modules. System test: all the modules. Acceptance test: defined by user/customer.

22

Terminology
Non-functional testing Performance test Load test Soak test

23

Non-functional testing: the “-ilities”. Performance test: how fast. Load test: how many. Soak test: how long.

23

Terminology
Continuous integration

24

This is an alternative to “big bang integration”, otherwise known as “integration hell”.

24

Tools

Mock Objects Unit Tests

jMock, EasyMock JUnit, TestNG

Build Ant, Maven IDE Eclipse, NetBeans

25

25

Tools

Web Apps JEE Servers Databases

Selenium, HttpUnit Cactus DBUnit

26

HttpUnit is better suited to situations where you want to examine the DOM at a low level. If you're looking for convenience, and don't need to get deep into the details, look at Selenium instead.

26

Tools

Acceptance Selenium, FitNesse, Stiq Performance JMeter, Eclipse TPTP Coverage Cobertura, Emma

27

Selenium IDE, a Firefox plugin, lets users or product managers create their own scenarios. Selenium is also useful during development, to quickly fast forward through actions to get the web application into a desired state.

27

Tools

Continuous Cruise Control, Integration Continuum, Hudson

28

28

Tools
Continuous Integration Acceptance Performance Coverage Cruise Control, Continuum, Hudson Selenium, FitNesse, Stiq JMeter, Eclipse TPTP Cobertura, Emma Selenium, HttpUnit Cactus DBUnit jMock, EasyMock JUnit, TestNG Ant, Maven Eclipse, NetBeans
29

Unit testing

Web Apps JEE Servers Databases Mock Objects Unit Tests Build IDE

The items in green aren't really a stack, just a collection of technology-specific libraries that can be useful for creating test cases.

29

Demo
Tools

Be sure to design Selenium tests so they can be repeatable. Either delete data when they are done, or always start from a fixed known state.

30

Resources: SGV JUG
Web Site
http://sgvlug.net/mailman/listinfo/java-sig

Mailing List (Hosted by SGV Linux Users Group)
http://sgvlug.net/mailman/listinfo/java-sig java-sig@sgvlug.net

31

31

Resources: TDD
Articles
http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)

32

32

Resources: IDE & Build Tools
Eclipse IDE
http://www.eclipse.org/

NetBeans IDE
http://www.netbeans.org/

Ant
http://ant.apache.org/

Maven 2
http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM

Maven & Eclipse
http://m2eclipse.codehaus.org/

Maven & Netbean
http://maven.apache.org/guides/mini/guide-ide-netbeans/guide-ide-netbeans.html
33

33

Resources: Unit Testing
JUnit Framework
http://junit.org/

JUnit FAQ
http://junit.sourceforge.net/doc/faq/faq.htm

JUnit with Ant
http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/manual/OptionalTasks/junitreport.html

JUnit with Maven
http://maven.apache.org/plugins/maven-surefire-plugin/

TestNG Framework
http://testng.org/doc/

Jester – Mutation Testing
http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
34

34

Resources: Mock Objects
Mock Objects Blog
http://mockobjects.com/

EasyMock Library
http://easymock.org/

jMock Library
http://jmock.org/

35

35

Resources: Unit & Functional
XmlUnit Library
http://xmlunit.sourceforge.net/

DBUnit Library
http://dbunit.sourceforge.net/

HttpUnit Library
http://httpunit.sourceforge.net/

Cactus for JEE Server Code
http://jakarta.apache.org/cactus/

Cargo for JEE Server Code
http://cargo.codehaus.org/

36

36

Resources: Test Coverage
Cobertura
http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/

Cobertura with Ant
http://cobertura.sourceforge.net/anttaskreference.html

Cobertura with Maven
http://mojo.codehaus.org/cobertura-maven-plugin/usage.html

Emma
http://emma.sourceforge.net/

37

37

Resources: Perf. & Load
JUnitPerf
http://clarkware.com/software/JUnitPerf.html

JMeter
http://jakarta.apache.org/jmeter/

Eclipse TPTP (Test & Performance Tools Platform)
http://www.eclipse.org/tptp/

Netbeans Profiler
http://profiler.netbeans.org/

38

38

Resources: Acceptance Tests
Selenium for Web Apps
http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action

Automating Selenium (“Remote Control”)
http://www.openqa.org/selenium-rc/
http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration

Selenium & Maven
http://raibledesigns.com/rd/entry/selenium_plugin_for_maven

FitNesse
http://fitnesse.org/

Stiq
http://storytestiq.sourceforge.net/

39

39

Resources: Cont. Integration
Cruise Control
http://cruisecontrol.sourceforge.net/

Continuum
http://maven.apache.org/continuum/

Hudson
https://hudson.dev.java.net/

40

40

Resources: App used in Demo
Open Source Medical Record System
http://www.openmrs.org/

Prerequisites: SVN, MySQL 5, Tomcat 5
http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/

SVN Repository
http://svn.openmrs.org/openmrs/branches/alpha

Installation
Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
41

41

Thank you
To all who participated!

42