Automated performance testing using Maven & JMeter

George Barnett, Atlassian Software Systems @georgebarnett

 Create controllable JMeter tests Congure Maven to create a repeatable cycle Run this build in your CI server every 8 hours Find performance regressions faster

WHY?
Catch regressions quickly Stop hunting for bugs Code with condence Know the performance cost of features Fearless external dependency upgrades Go home on time

ATLASSIAN
15,000+ customers 100+ developers 8 software streams 1 performance engineer

ATLASSIAN
JIRA - Issue management Conuence - Enterprise Wiki Bamboo - Continuous Integration FishEye, Crucible & Clover - Dev Tools Crowd - Single Sign-On.

BEFORE WE BEGIN
Continuous Integration is critical Keep code working with Unit and Functional
tests

Broken or irregular builds means no code to

performance test

Performance testing is NOT a replacement

for other testing (but it helps)

PROCESS
Write code (1-6 months) Compile code Send it to QA (or Production?!) Performance Regression PANIC!
No performance testing!

A BETTER WAY
Write code CI compiles code and runs tests Send it to QA (or Production?!) Performance Regression PANIC!
No performance testing!

A BETTER WAY 2.0

Performance test artifacts as soon as they
pass Unit and Functional testing

Fast feedback for developers No Surprises

TOOLS
JMeter Maven & plugins Bamboo & plugins A Proler

JMETER
Apache Project Java Supports HTTP, JDBC, SOAP, XML-RPC,
FTP, SMTP, LDAP, JUnit & more.

MAVEN
Apache Project Java Build Tool Dependency management Documentation

MAVEN CONCEPTS
Project Object Model (POM) Standardised directory structure Build Lifecycle Reporting Lifecycle Plugins

MAVEN POM
<project> <groupid>com.atlassian.com.performance.jira</groupid> <artifactid>automated-performance</artifactid> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <build> <!--Main Lifecycle plugins congured here--> </build> <reporting> <!--Reporting Lifecycle here--> </reporting> </project>

MAVEN PLUGINS
Chronos Ant-run Cargo Maven Assembly plugin Maven Dependency plugin

BAMBOO
Continuous Integration Can run builds on Amazons EC2 JMeter Aggregator Plugin

A PROFILER
Java Proler CPU Snapshots GC & Memory information Thread usage and contention Triggers & ofine sampling

STEP 1
Package the JMeter test

Maven Assembly plugin Create a .zip from project directories `mvn deploy` to repository as a SNAPSHOT

STEP 2
Congure the Maven automated build

Package application state into zip les Ant-run - application pre-setup Cargo - deploy container Chronos - congure the JMeter tests

MAVEN PROFILES
<project> .. <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> </properties> <prole> <id>dataset1</id> <properties> <test.dataset>/opt/dataset/dataset1.zip</test.dataset> .. <prole> <id>dataset2</id> <properties> <test.dataset>/opt/dataset/dataset2.zip</test.dataset> ..

MAVEN PROFILES
Invoked on command line Conguration in prole is added to build Use for a wider variation in test
environments JMeter

eg: mvn verify -Pmysql, tomcat5, dataset1

Use proles for individual work units, eg

ADDING A PROFILER
$ export LD_LIBRARY_PATH=$JPHOME/bin/linux-x64/ <global.jvm.args>-Xms2000m -Xloggc:${project.build.directory}/gc.log</global.jvm.args> <jvm.args>${global.jvm.args}</jvm.args> <prole> <id>jproler</id> <properties> <jvm.args>-agentlib:jprolerti=ofine,id=${jproler.proleid},cong=${jproler.congle} Xbootclasspath/a:${jproler.home}/bin/agent.jar ${global.jvm.args}</jvm.args> </properties> </prole>

REPORTING
CHRONOS

Available in maven Generates graphs and a numerical report Can compare to historical data Static :(

eg: mvn site / mvn chronos:report

REPORTING
CHRONOS

REPORTING
CHRONOS

REPORTING
CHRONOS

REPORTING
CHRONOS

REPORTING
CHRONOS - HISTORY

REPORTING
CHRONOS - HISTORY

REPORTING
BAMBOO JMETER AGGREGATOR

Open source Bamboo plugin (supported!) James Roper (Atlassian) 20% time project Can graph JMeter .JTL and CSV les Dynamic

REPORTING
BAMBOO JMETER AGGREGATOR

TIPS
Allocate time for xing tests Your tests WILL break Work out a process for xing bugs Gather as many artifacts from each test run
as possible

React sooner rather than later

Q&A
gbarnett @ atlassian . com @georgebarnett Code samples online soon! blogs.atlassian.com/developer Meet Atlassian @ The Summit: May 31 - June 2 in SF www.atlassian.com/summit