Load Testing Approach

Quick Guide to Plan and

Execute Load Test Using
Load Runner!

-Ranjit
jcrvs@hotmail.com
Why Is a Approach
Needed?
• Quality approach
work smarter to be
faster.
• Do it right first
time, every time.
• Detailed approach
that prevent from
getting lost in the
“load testing sea”.
• Use of smart tools
load runner.
Conventional Approach
• Random
requirements from
management fro
load test
• In-experienced
staff
• No requirements
analysis performed
for load test
• Easy to get carried
away in load
testing
The Approach
• Perform requirement
analysis to get
information as
– Type of load test
– User load estimation
– User load distribution
– User activity analysis
– Production
environment analysis
– Database size
– Report generation
– Simple metric
collection
Type of Load Test
• Determine the goal
of the test
– Do we intent to find
the synchronization
problems?
– Do we intend to
perform a load test?
Or a stress test? Or
a performance test?
Synchronization Problem
Tracing Load Test
• Goal of this test is to determine the
synchronous issues and trouble some
areas.
• This test may need vu-generated scripts
with lots of rendezvous points or
rendezvous points at each post/action.
Load/stress/performance
• Different type of tests :
– Load test determines how is the performance of
application under the concurrent user sessions
for typical user scenario. The think time taken
into consideration in these test scripts.
– Stress test examines how application behaves
under maximum load. In simple terms find the
upper threshold for the application below which
it can work normally. Think time ignored in these
tests.
– Performance test indicates response time for the
entire application from the user’s perspective.
What Do U Want?
• Do determine what
kind of test do you
want.
• Then plan ahead.
User Load Estimation
• A detailed feed back form the
marketing/business
development will give a idea
of the user load or the
number of users using the
product. This will determine
the load to be used against
the product in testing.
• An inexperienced staff may
configure a load test to
simulate 1000 v-user but user
base for the application may
be not more than 400. This
may result in licenses being
lost for the v-user and time
and efforts as well.
User Load Distribution
• A detailed feed back
form the
marketing/business
development will give a
idea of the user load
distribution.
• The user load may be
peak in the morning or
afternoon and very less
in the evening.
• This may also help
determine the
concurrent users and
simultaneous users.
User Load Distribution
• This factor will input to
the scenarios to be
used and configured in
the load test.
• E.G. BD may come up
with a user load of
1000 users per day and
each grouped to
perform certain activity
as say 30% in section A
of application rest in
section B of the
application etc.
User Activity Analysis
• A detailed discussion with
marketing/BD may reveal
the user activity details
on which detailed scripts
can be written.
• E.G. In morning there may
be 200 forms submitted or
in evening most of the
users may login and
perform results
generation activity so in
morning scenario out of
1000 concurrent user,
70% may be performing
this activity.
Production Environment
Analysis
• A discussion with the
I.T. Department will
throw light on
deployment
environment.
• This may input valuable
information as does the
environment have
enough hardware? Is it
running in a clustered
environment?
Production Environment
Analysis
• Mirror the
production
environment into a
test bed.
• Alternatively
create test bed and
after load test
suggest production
environment to
management.
Database Size
• The database size does matter in the
load test.
• Bulky the database, more the
latency.
• Define the database size prior to load
test e.G. Conducting first set of test
on a 50K sized DB then a 100K sized
DB and so forth.
Report Generation
• Generate two types of reports
– Load runner detailed reports for
engineering department
– Generic reports for the activities
performed in the scenario and response
time details with other observation and
conclusions for the management
• Generating graphs also helps
– Helps in tracking per load cycle results
– Easy interpretation for people
Simple Metric Collection
• Basic metrics can be
100 collected as follows:
CPU – CPU utilization :
80
Utiliza should not exceed
tion 60%.
60
Total – Per page size. This
40 is a indicator of the
hits
bulky-ness of the
20 page. Response time
Per for each transaction.
0 page – Configuration
1st 4th byte details.
Cycle Cycle – Test bed details.
Simple Metric Collection
– Total hits and
100 hits/second, should
Memor not be greater than
80 20 or request queue
y
details need to be
60 collected.
Proces – % Failed
40 ses transactions, should
20 not be greater than
Failed 5%.
0 transa – Number of
1stCycle ctions processes running
on server.
– Memory details.
Simple Metric Collection
• Load runner can be used to collect the
above metrics.
• Other option is to use O.S. Specific tools.
Plan on the Above Analysis –
Clubbing All Together
• After the requirement analysis, feed all the
data into a test plan.
• Test plan will contain the type of test to
perform, test bed details, load/user details
and activity scenarios.
Plan on Scenarios
• Plan scenarios for execution based on the
above input and different daytime for the
day as morning scenario, afternoon
scenario etc.
• Each scenario may scale to 3 to 4 hours. So
activity analysis as number of forms
submitted per day etc and time may help to
set the number of iteration for each
scenario.
Follow up Action:scripts
• Create test scripts with vu-generator for
the interactive activities to be performed
with application.
• Unit test in vu-generator debug mode.
• Also run the same in 10 user 10 iteration
load runner scenario mode. Use the option
“show v-user” to check for the activities
being performed.
• Parameterize scripts and co-related them.
• Make scripts dynamic, more efforts to be
put in for making dynamic than just co-
relating the scripts.
More on Scripts
• Add comments and queries to fetch the
parameterized data from database to make
it user independent.
• Logs may reveal error which at time may
be hidden.
• In runtime settings, use option as mark
each step as transaction to prevent adding
manually transaction function for each
step.
Configuring Scenarios
• Recommended to unique create groups, each group
is represented by one script.
• Use ramp-up and ramp-down features to prevent
instantaneous loads on load test server. Load
runner gives one click access to pre-configured
“slow ramp-up” and “ramp up” scenario.
• If scripts are dynamic then recommended to use
duration based scenario.
• More refined control can be achieved for scenario
by scheduling by group rather than scheduling by
scenario.
Configuring Scenarios
• Enable standard logging.
• One click generator configuration can be performed
using load runner.
• In runtime settings, use option as mark each step
as transaction to prevent adding manually
transaction function for each step.
• The think time in-between two iterations is
recommended in load tests.
• Proxy may or may not be needed. Recommended not
to use proxy to get a benchmark results first, then
run the same scripts though proxy and compare
results.
Configuring Scenarios
• Simulating browser cache option may be
turned off to make sure each request is
fetched form the server.
• Bandwidth throttling may or may not be
used as per requirements.
• Enable web performance graphs option in
load runner to monitor at test runtime the
results.
Configuring Scenarios
• Disable or enable “continue on error”
option. Recommended to enable it for
cluster fail-over tests and also regular load
test.
• The option to run v-user as a process or
thread may be used as per requirements.
Recommended to use option is thread to
prevent overhead of the memory and
system resources. But also more threads
per process makes the process unstable.
Load runner allows configuration of number
of thread per process for stability.
Execute Tests and Report
• Prepare test bed
and execute
planned scenarios.
• Recommended
server restart
before each test
cycle.
• Generate reports
for each cycle.
In a Nutshell
• Planned approach
make life simple and
makes load test simple
too.
• Requirement analysis
is important activity
before starting load
test.
• Metrics are the end
results so save results
for each cycle and
analyze.
• FLY HIGH with simple
minds and simple
practices !!
Glossary
• LR- load runner
• O.S. – Operating system
• BD – business development
Q&A