Load Testing with JMeter

Larry Davidson October 21, 2008

What is Load Testing?

2

User

User

User

User

App
3

Normal Use Few Concurrent

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

App
4

Spike Use Many Concurrent

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

User

App
5

Spike Use Many Concurrent

Load Testing
Key goals
Can app support expected concurrent users? At what load does the app break?

Steps
Identify primary user paths Identify expected concurrent users
Both now and down the road

Set up virtual users to hit the app Run the test Analyze the results

Load Testing Approaches

Open Source Tools
JMeter ab http_load The Grinder Siege

Commercial Tools
LoadRunner (around $200,000 to own!) WebLoad Third-party $ervices

Why We Chose JMeter for Internal Testing

Easy to install and use
Free! Java most platforms GUI vs. command line Just download and run!

Feature-rich
Post forms Record from browser Load test data from les Add logic, variables & functions Run one test from multiple machines Test many protocols, not just HTTP
8

Installing & Running JMeter

1. Download from http://jakarta.apache.org/jmeter! 2. Unzip (path with no spaces preferred) 3. Go to "bin" directory 4. Run appropriate le
Windows:! Double-click jmeter.bat Mac:!!!!!!!!! Double-click ApacheJMeter.jar Unix/Mac:! Run ./jmeter from command line
9

Looks complicated ... but it's not!

10

Using JMeter's GUI

Hierarchy based! "Test Plan" is root element
Add test elements as children Right-click and choose "Add > ..." Give items meaningful names

"WorkBench" is for non-test helpers

HTTP Proxy recording
11

Modify properties in right panel

Properties saved after leaving entry elds

What We're Trying to Do

Simulate users
Dened numbers of concurrent users Dened period of time Various user types (normal, registered, admin, etc.)

Simulate web requests

Specic web pages Form submissions Alternate or random paths
12

Analyze reports
Response times acceptable? Handles expected load? What load causes failure?

JMeter's Nomenclature
User!!!!!!!!!!!!!!!!!! Thread Request!!!!!!!!!!!! Sampler Report!!!!!!!!!!!!!!! Listener
13

Let's run a Load Test already!

14

Setting Up Your First Load Test

Simulate users
Add > Thread Groups
Number of Threads (concurrent users) = 10 Rampup Time (seconds until users active) = 1 Loop Count (how many repetitions) = 1

Simulate web requests

Add > Sampler > HTTP Request
Server Name or IP = www.yahoo.com Path = /
15

Analyze reports
Add > Listener > Aggregate Graph Add > Listener > View Results Tree!

Running Your First Load Test

Choose "Run > Start" (ctrl-R) Click on the "Aggregate Graph"
Data lls in as test runs Click "Display Graph" for chart
47 threads (users) Running!

Watch upper right square

Green = running, Gray = done 47/100 means 47 of 100 threads are running
16

To run again with clean data...

Choose "Run > Clear All" (ctrl-E) Choose "Run > Start" (ctrl-R)

Verify Details with View Results Tree

Details for each request, good for troubleshooting Sampler Result
Thread number and details Response times Server response codes and headers

Request
Request sent to web server
17

Response Data
Server response body (HTML, etc.) View as text or rendered HTML, etc.!

Let's get more advanced!

18

A Test with Three User Types

Users can do various things on a site
Good to simulate several common tasks at same time

For this demo, we"ll test 3 users

1. Yahoo Viewer 2. Google Viewer 3. CNN Viewer

To start, add 3 thread groups under Test Plan

Set to run all concurrently (this is default) Add HTTP Request to each, with appropriate URLs

19

Add Listener under Test Plan and Run

Notice that all 3 URLs are requested at once

Logic Controllers
Group Items
Simple

Choose One Per Loop or Thread Group

Interleave, Random, Switch*

!Looping
Loop, ForEach*, While*

Logical*
If

20

Time Control
Throughput, Runtime!

Randomly Selecting From 3 URLs

Let"s modify the 3 user types Test Plan Move the 3 requests under one Thread Group
1. Yahoo 2. Google 3. CNN

Choose all 3, right click, choose Insert Parent...

Choose Controller > Random Controller
21

Add Listener under Test Plan and Run

You"ll see that only one random request is chosen

Add an If Controller
Let"s modify the existing Test Plan Add Controller > If Controller under Thread Group Copy the 3 requests
1. Yahoo 2. Google 3. CNN

Paste under the If Controller Delete the Random Controller

22

Let's make a recording!

23

Recording your load test

Add Proxy Server to WorkBench
Non-test Elements > HTTP Proxy Server Choose a "Target Controller" for recorded data! Click "Start"

Set browser to use Proxy Server

Edit browser's Network Settings
Usually under Preferences > Advanced > Network/Proxy Proxy = localhost; Port = 8080
24

Browse and record Click "Stop", then edit recorded requests

Recording tips and tricks

Limit what's recorded
Enter "text/html" in Content-Type lter (Include) Enter *.gif, *.css, etc. in URL Patterns to In/Exclude!

Use Grouping if recording multiple pages Add HTTP Request Defaults under Controller
Fill in domain Now matching recorded domains will be blank
25

Record time between page requests

Add "Constant Timer" under Proxy! Enter "${T}" in delay eld to record actual time

Recording forms
JMeter records all submitted elds
Easier than examining source code or Firebug!

JMeter also records browser headers

Some pages (e.g., Google) require valid browsers

Remember to accommodate cookies if needed!

add Cong Element > HTTP Cookie Manager

Edit eld values after they're recorded

OR ... why not simulate varying user data!

26

Let's add variables!

27

JMeter Variables
Dene global variables
Test Plan (root element) Cong Element > User Dened Variables

Dene virtual user variables

Preprocessors > User Parameters Cong Element > CSV Data Set Cong

Reference as ${variableName}
Can be used in nearly any property Scope determined by location in hierarchy

28

Variables from a CSV le

Create a CSV le with the data you want to use
No label row; save in same folder as .jmx le

Add "Cong Element > CSV Data Set Cong"

Add under the Thread Group that will use it Set Filename = lename.csv Set Variable names = username,password (e.g.)

Set variables in HTTP Request "Send Params"

Name = username, Value = ${username} Name = password, Value = ${password}

29

Will read in data with each new thread, or with loops

Debugging
Add Debug Sampler
Set JMeter variables as "True", others "False"

Add View Results Tree Listener Run Click View Results Tree Click Debug Sampler in Results Tree
Click "Response data" tab to view variables Looped user vars will show with numbers
30

JMeter Functions
Referenced as ${__functionName(properties)}
Like variables, can use in JMeter properties

Use Options > Function Helper Dialog

Choose a function Enter parameter values Click "Generate", then copy/paste function

Helpful functions
${__javaScript(expression to evaluate)} ${__threadNum()} ${__Random(min, max, varname)} ${__regexFunction(regEx, template, match, ...)}

31

So what about reports?

32

Reading Reports
Most times are reported in milliseconds Throughput
Requests per second

90% Line
Value below which 90% of request times fall

Standard Deviation
A measure of certainty / uncertainty Measures variation from average

33

Latency
Delay between request and response

Response Time Goals to Aim For

0.1 second perceived instant 1.0 second uninterrupted ow 10 seconds user loses interest

Going further...
35

Higher loads with multiple "slave" computers

Install JMeter on other computers Find the IP addresses of the other computers Start JMeter server on all computers
Windows: Double-click [jmeter]/bin/jmeter-server.bat Mac command line: cd [jmeter]; ./jmeter-server

Add IP addresses to master computer's JMeter

Open [jmeter]/bin/jmeter.properties in a text editor Add IPs to remote_hosts=127.0.0.1,192.168.1.241

36

Choose "Run > Remote Start All"

Limitations of JMeter
Can't run embedded Javascript, Flash, etc.
But ... use recording to capture those requests

Limited by your computer's network connection

You may have to work with IT to optimize

1 computer limited to about 300 concurrent users

But... use multiple slave computers to handle more

!Reports are a bit lacking

But... save results and import into Excel, etc.

37

Reference

http://jakarta.apache.org/jmeter

38