You are on page 1of 8

FUNDAMENTALS OF LOAD TESTING

Introduction:

Load testing is required at every stage in the life cycle of an application, in particular web
applications. Load tests help identify performance problems during system design and
development stages. For existing applications, the simulation becomes an aid to maintain, fine-
tune and forecast future system needs.

Abstract:

In simple context, “Load testing” is the simulation of real world loads on computer systems. A load
test is the one that takes into consideration – the various # of users that execute tasks on the
system and the various types of software on users’ machines, including processes, browsers,
connection speeds etc. So, before going into production, load testing is the only way to evaluate
how the system will perform in the real world. There are various ways to proceed with a Load test
and in general load testing is to – identify and analyze system load requirements, define a “site
map”, identify features to load test, define user sets, record tests with single user, randomize
data, select and adjust work load - virtual users (user sets) and load agents, run test and analyze
test.

This write up will give you an overview or fundamentals of Load testing and will not cover any
advanced concepts of load testing. It covers Load testing details of a web application, Benefits of
load testing, Load testing environment, Load test types & variations, Basic work load models,
Quick note on how to select a work load model, Load testing goals and Results and list of well-
known Load testing tools.

A Load Test by definition:

• Emulates real-life interaction between users and server systems


• Creates a model of what a typical user does within the application and measures the
performance and reliability of the server
• Monitors and quantifies the performance of an internet application under expected and
peak user conditions
• Measures the stability, responsiveness and throughput of the computer system in a
realistic test environment
• Generates different types of traffic expected, in the volumes expected, using actual
computer hardware and software
• Brings the entire functionality of the e-business web site under scrutiny in order to identify
the general areas of possible bottleneck occurrences
• Measures the performance of the web servers and/or the firewalls
• Allows testers to draw useful conclusions from the test results for improving the site's
components
• Finally, it is entirely different from functional and regression testing. Functional testing
determines whether the system operates as expected in general - a single user mode
and Regression testing determines whether an upgraded version operates as expected,
but load testing determines whether system operates as expected when used by 100,
200, or a 1000 or more simultaneous users.
Benefits of Load Testing:

• Minimize the risk of deploying systems that don't meet quality and performance
requirements
• Find the root cause of performance bottlenecks and system failures with ease
• Optimize user experience by testing against service levels to ensure that Service Level
Agreements are met in production
• Reduce the costs of defects by testing remote components early in the development
cycle
• Minimize hardware and software costs by accurately predicting system capacity
• Provides performance and capacity statistics
• Tests scalability of the site
• Allows you to evaluate the bandwidth for the amount of data sent and received
• Allows you to verify predicted performance under varying conditions
• Verifies the tuning of your system through benchmarking

Typical Load Testing Environment:

Load testing takes place in a client/server environment. The attributes of the client and server
systems will impact your load test design. The basic components of a load testing environment
are

1) Users – are individuals who use an application


2) Client – is a combination of hardware and software used by the user (browsers,
network, connections, etc)
3) Server – refers to system that provide service requests to clients – application
servers, database servers

The below diagram shows a load-testing environment for a typical web application and a number
of driver machines, connected together through a local and/or wide area network.

Three conditions must be met before load testing can begin:


1) the load-testing environment – the Web application to be tested, the network, and the
load-driving machines – needs to be adequately equipped
2) the virtual users must be set up to perform tasks in the same way as real users
3) the Web browser emulation used must be as accurate as possible
Preparation before Load Tests:

First you need to analyze the “application under test” and determine the factors which affect the
application performance and mark them as your “Key features or attributes” to measure like
Server side – CPU utilization, memory usage, process time, database server/queries and Client
side – similar Health monitoring data, network, user profiles and most importantly application data
and transaction points.

Apart from the above and a well-defined Load testing methodology/process has to be determined
to get satisfactory load test results i.e., identify and analyze system load requirements, define a
“site map*”, identify features to load test, define user Action sets* (test scenarios), identify user
groups, user profiles, form user types, record tests/script load test for single user, randomize
data, form user types* - user profiles* vs. user groups*, adjust work load – select work load model
and choose # of users, run test and analyze test.

*Site map – A navigational flow chart/map which identifies all of the possible paths that a user can
navigate within an application.
*Action sets – defines the actions that typical users perform when using the application, these can
be scripted as functions and best used for “Isolation” type of load tests
*User profile – users distinguished or categorized by the software or hardware configuration they
use like users with IE browser with Modem connection, users with Netscape navigator with High
speed broadband connections etc. Basically they are Browsers type vs. Browser version vs.
Network connection.
*User group –These are combined with action sets to form
*User types – are formed when User profiles and User groups are multiplied (permutations n
combinations of both)

You can pin-point the exact bottleneck of an Application under load test by choosing the right
combination of a User type and Action set.

NOTE: However predicting sever-side errors is quite tricky, it needs reference to wide range of
result reports to pin-point the exact bottleneck like network behavior reports, database (if any)
server behavior, behavior of each network component like proxy, firewall, switches, router etc.

So, “Load testing methodology” can be summarized as below:

• Derive Gather statistics to get the feature list most accessed by customers
• Identify load test scenarios
• Generate load test plan for each test scenario
• Generate load test script for test plan derived
• Try script for single virtual user
• Customize session variables and randomized data variables
• Prepare required input Test data files
• Run script with required workload configuration
• Analyze load test reports generated
Basic Types of Load Testing:

To make an accurate diagnosis of a performance problem in an e-business application, it is


usually necessary to run a sequence of tests that progressively focus in on the particular problem.

1. End-to-end Testing
• Is generally the first type of testing executed
• Consists of a global examination of the entire Web application
• Is designed to bring the entire functionality of the Web application under scrutiny in order
to identify the general areas where bottlenecks are
• Can use a variety of load test types and workload types depending on the context – most
often, stress tests or stability tests, deploying constant or increasing numbers of virtual
users are used

2. Module Testing
• Typically follows the end-to-end testing
• Separately tests specific modules of an application to accurately pinpoint any problems
• Generally uses isolation tests that are designed to bombard the suspect module with
loads large enough to force repeated occurrences of a particular problem
• Generate very specific and detailed information, allowing the tester to accurately
diagnose the problem

Load Test Variations:

Load tests are designed in different ways, depending on what they are required to measure. The
most frequently used load tests are stress tests, stability tests, and isolation tests.
1) Stress Test
• Determines how much load a system can handle
• Determines the maximum number of requests a server can manage
• Generates a large and continuous load to stress the system to its limit
• Minimizes the time between transactions
• Ignores think times
• May be set using the profile or by running a try script
2) Stability Test
• Determines whether a system will remain stable over a long period of time
• May last 24 hours, a week or even longer
• Mimics the real traffic a system will encounter
• Determines whether response time will remain stable over a long period of time
• Determines whether resource consumption will remain stable over a long period
of time (i.e. memory leaks)
• Helps identify other technical faults that may compromise the stability of the
system
3) Isolation Test
• Helps identify bottlenecks in specific areas of the application
• To isolate a troublesome functional area & find flaws by repeated load testing
• Often executes specific sets of transactions over and over again
Load Testing Basic Work Load Models:

A work load model In general any load testing tool will provide the following work load models.

ƒ Increasing - An increasing workload helps testers to find the limit of a Web application’s
work capacity, and is thus crucial for establishing fail-safe conditions. This workload
model is especially useful when you want to find out at which load level your system
crashes or does not respond within acceptable response times

ƒ Steady State - In this model, the same number of virtual users is employed throughout
the test, and there is no delay between transactions. This workload model is especially
useful when you want to find out about the behavior of your tested system at a specific
load level. The use of steady-state workload provides exact response time and
throughput measurements for a given number of users. This type of workload is often
employed for stress and stability tests

ƒ All Day - You can assign different numbers of virtual users to any interval of the load test.
Each user type can use a different load distribution. Therefore, you can design very
complex workload scenarios such as workday workloads or even weekly workloads. As in
the dynamic workload model, you can adjust the load level during a load test. This
workload model is especially useful when you want to model complex, long lasting
workload scenarios in the most realistic way possible

ƒ Queuing - In this model, transactions are scheduled following a prescribed arrival rate.
The load test will finish when all of the virtual users have completed their prescribed
tasks. Note that the test may take longer than the simulation time that was specified,
because of the randomized arrival rates.
ƒ Dynamic - In this model, you can manually change the number of virtual users in the test
while it is being run. The maximum number of virtual users to be run is set; within this
limit; the number can be increased or decreased at any time during the test. No
simulation time is specified, and you must bring the test to an end manually. This
workload model is especially useful when you want to experiment with different load
levels and to have the control over the load level during a load test.

The graph can’t be provided since load is manually controlled.

How to identify the workload model & Best practice:

Review your scenario question. Usually, the language in the question helps you determine which
workload model to use.

See the table below:

Language in Scenario/ Question Workload Model


Support Increasing
Stable, Memory Leaks Stead state
Respond to morning/afternoon spikes All Day
Response when 1000 users perform the same Either “Stead state” with the wait for
action simultaneously function or “Queuing”.

Best practice: When workload requirements are not given as part of your product specifications,
go blindly with a “increasing workload” model when you know it will be used by multiple and
simultaneous users. So, use 10 % users as initial user with 10% user increment at 10% of total
test time as the increasing period.
Load Testing Goals & Results:

The goal of any load test should be clearly understood and documented. A load test usually fits
into one of the following categories:

1. Quantification of risk - Determine, through formal testing, the likelihood that system
performance will meet the formal stated performance expectations of stakeholders, such
as response time requirements under given levels of load. This is a traditional Quality
Assurance (QA) type test. Note that load testing does not mitigate risk directly, but
through identification and quantification of risk, presents tuning opportunities and an
impetus for remediation that will mitigate risk.
2. Determination of minimum configuration - Determine, through formal testing, the
minimum configuration that will allow the system to meet the formal stated performance
expectations of stakeholders - so that extraneous hardware, software and the associated
cost of ownership can be minimized. This is a Business Technology Optimization (BTO)
type test

To speak in load testing tool terms, load testing is to

• Determine each Transaction* “Response time*” under specific load


• Determine each user “Session time*“ under specific load
• Determine “Transactions/sec”* within Load test simulation time
• Determine “Errors/sec*” within Load test simulation time
• Determine application overall “Throughput*” during Load test simulation time

Finally, analyzing all the above Load test results under various Loads and determine how much
load (maximum # of users) your “Application” can withstand without its performance getting
degraded. Depending on these results, you can plan how your production environment should
like and scale it up whenever needed.

*Transaction – a unique business query/event matching application requirements


*Response time – it is the time taken from a user initiating a request to the instant at which the
first part of the response is received at by the application.
*Session time – it is the time taken by the application for each user to execute all transactions
specified under load test scenario
*Transactions/sec – generally rate of no. of successful transactions per sec. Might differ for each
load test tool, for few it is categorized as TransacationOK/sec which means successful
transactions and Transactions/sec might mean successful and failed transaction rate.
*Errors/sec – No. of errors load test generated during simulation time. It need not be only
transaction failures but can be actual errors application has generated (bugs), script errors
mismatched with actual result, network issues etc.
*Throughput – it is the overall transaction time taken from a user initiating a request to the instant
at which the last part of the response is received at by the application
Major Load Testing Tools (in the 93% of Market Share)

• HP’s (earlier MERCURY INTERACTIVE) - LoadRunner [54%]


• IBM RATIONAL’s Performance Tester [11%]
• COMPUWARE’s QA load [9%]
• BORLAND’s (earlier SEGUE) - Silk Performer - 7%
• EMPIRIX’s e-TEST suite [6%]
• RADVIEW’s WebLoad [3%]

Conclusion:

“Load testing” is the only way and plays an important role in analyzing the system performance
for a specific load (no. of simultaneous users). Before an application is planned to go “public” -
typically a client/server application where the no. of users accessing it are larger in number,
unpredictable and random, there is a “desperate” need to perform load testing and define the
“maximum load/simultaneous users” it can sustain without its performance getting degraded.
Once this performance measured is “defined” as a requirement, you can definitely plan to scale
“up” your production environment to meet whatever additional load of users that keep
bombarding your application.

References:

Fundamentals of Load testing – Search links from Google


SilkPerformer – Modeling & Implementing Load tests student guide (Segue)