Performance Testing Using Load Runner

1
Load Runner
TABLE OF CONTENTS
Introduction 5
• Definition for Performance testing 5

• Types of Performance Testing 5

• Need for Performance testing 5

• Automate Performance Testing 6

• Automated Performance Testing Phases 6

Requirements collection 7

• Design 7

• Build 7

• Execute 8

• Analysis and Tuning 8

Performance Test Planning 10

• Load / Performance Tool Evaluation & Selection 10

• Test Process / Methodology and Test Strategy 11

• Performance Test Start/Stop Criteria 11

• Test Environmental Setup and Pre-Requisites 12

• Analysis and Report Generation 12

• Pass / Fail / Exit Criteria 12

Common Implementation Issues 13

• Preparation 13

• Test Environment Setup 13

• Customization 14

• Execution/Analysis 15
Introduction to Load Runner 16

• Load Runner Terminology 17

• Load Runner Vuser Technology 17

Creating Scripts Using Load Runner 18

• Recording Vuser Scripts using VuGen 18

• Enhancing the scripts 19

2
Load Runner
 • Configure Run time Settings 20

Executing Scenario using Controller 29

• Scheduling a Scenario 29

• Setting Up Load Generator Machines 30

• Configuring Load Generators 30

• Setting up the Monitors 31

Analyzing Results 33

• Summary Report 34

• Running Vusers Graph 35

• Hits per Second 36

• Transaction Summary 37

• Average Transaction Response Time 38

Tuning 39

• Information Captured for Tuning 39

• Performance Counters to be captured 39

3
Load Runner
Abstract
The incredible pace of change and the explosion of software complexity introduce
tremendous risk into software development process. Rigorous testing is the most
common strategy to quantify and reduce this risk to the business. The question for
developers, QA teams, and management alike is how accurately and thoroughly they
can validate the system’s performance before Go Live – without breaking the Budget.

By conducting performance testing, an organization can verify that the application(s)

meet the Business needs before Go Live. This ensures that there are no Deployment
Risks, and Quantifies the impact of changes on End-User’s experience by pinpointing
the Failed Components for Rapid Resolution.

This paper presents a complete practical approach for performance testing and
covers the best approach which includes the following:

• What is performance testing?

• Why should you automate it?
• What is the right process for performance testing?
• Who should be involved in a good performance test?
• What are the requirements need to be collected for performance Testing?
• How to plan for performance Testing?
• What are the common implementation issues faced during the performance
testing?
• How to conduct a performance testing using Load Runner?
o Creation of Scripts
o Enhancing Scripts
o Creation and Execution of Scenarios

4
Load Runner
 o Analyzing Results
o Tuning the application.

Introduction

What is Performance Testing?

Performance Testing is aimed at verifying the system’s performance requirements

like response time, Transactional throughput and no of concurrent users.
Performance testing is used to accurately measure the End-to-End performance of a
system prior to Go Live. It will identify the loop holes in Architectural Design which
helps to tune the application.
It includes the following:
♦ Emulating ‘n’ number of users interacting with the system using minimal
hardware.
♦ Measuring End-User’s Response time.
♦ Repeating the load consistently
♦ Monitoring the system components under controlled load.
♦ Providing robust analysis and reporting engines.

Types of Performance Testing

Load Testing:
To verify that system can handle the expected load, on deploying it under Real Time
conditions.

Stress Testing:
To identify the application’s Break Point, also to measures whether the application’s
environment is configured to handle the expected or potentially unexpected high
transactional volume.

Scalability testing:
Scalability testing integrates very well with performance testing. The purpose of
scalability testing is to determine whether the application automatically scales to
meet the growing user load.

5
Load Runner
Volume testing:
To verify the stability of the system with respect to handling large amounts of data
over extended period of time.

Need for Performance testing

To verify the below mentioned:

 Application’s response for the intended number of users

 Application’s maximum load resisting capacity

 Application’s capacity of handling the number of transactions required by the
Business
 Application’s stability under expected and unexpected user load
 Ensuring that users have proper response time on production

Automate Performance Testing

Automating Performance Test is a discipline that leverages people, process, and

technology to reduce the Risks of the Application. It is about applying production
workloads to pre-deployment systems while simultaneously measuring system
performance and End-User experience.
Below are the loopholes in manual Load Testing that leads to Automation.
• Difficult to measure the performance of the application accurately.
• Achieving the synchronization among the users.
• ‘n’ number of real time users are required to involve in Performance Testing
• Difficult to analyze and identify the results & bottlenecks.
• Increases the infrastructure cost.

Automated Performance Testing Phases

Planning/Design

Build

Execution
6
Load Runner
Analysis and Tuning
Requirements Collection

Design

This is the primary phase where team will be gathering the requirements of the
performance testing. Requirements can be Business, Technical, System and Team
requirements.

Business requirements includes

1. An Application Overview: Performance Team should have good

understanding of the Application prior to Testing

2. Business Processes List: Team should gather complete information about

list of key Business activities that will be performed by an End User on the
system.

3. Business Process Flows: Document, which contains step-by-step details of

the Business processes identified by Business Team.

4. Transaction List: List of Key activities with in the Business process that
need to be performed in unit time.

Technical requirements should be collected from the System Administrators and

Database Administrators, which include the following:

1. Complete Architecture of the Environment

2. How many no. of users that the system should support during normal and
peak hours.

3. How many no. of transactions per unit time should be processed by the
system

4. Response time for all the Business Processes.

5. What are the different system workloads that will be experienced in

production

7
Load Runner
 6. How the load balancing will be done during the testing

Build

This phase consists of automating the requirements collected during the design
phase. It includes

1. Scripting: Record the business processes gathered in the design phase into
Automated scripts.

2. Transactions: These are the points that measure the performance of a

particular key activity with in the business process.

3. Parameterization: Input data needs to be passed for the scripts for

performing the Business activity on an application.

4. Scenarios: The complete End-to-End Business Workflow, which simulates the

Real Time Environment.

5. Monitors: Decide which servers or machines need to be monitored.

Environment setup consists of assembling Hardware, Software and Data required for
executing a successful real load. This may involve working with the systems, DBA,
Operations and Business Teams.

Execute

Execution will be done in multiple phases. It consists of various types of testing like

Baseline Testing

This testing is done to ensure that both applications’ functionality & environment
setup for performance testing is proper. This will be done with 10 to 20 users.

Performance Tests

This includes running the scripts with Average and Maximum load with specified no
of transactions. We will be simulating the real time environment with different load
conditions using different Scenarios, Think Time, Pace Time and no. of users.

Benchmark Tests

These are designed to measure and compare the performance of each machine type,
environment or build of the application in ideal situation. These tests are run after

8
Load Runner
the system undergoes scalability testing to understand the performance impact of
different architectures.

Analysis and Tuning

During the performance testing we will be capturing all the details related to the
system like Response time and System Resources for identifying the major
bottlenecks of the system. After the bottlenecks are identified we have to tune the
system to improve the overall performance.

Performance Optimization

This quantifies to improve the performance of the system as measured by improved

end-user response time or by reducing the overall required hardware infrastructure.

Performance Test Planning

To make any operation successful, planning plays a major role. According to the
80-20 theories, 80% of the time should be spent in planning and 20% in real
time plan execution/implementation. In the similar procedure, software
performance test planning is very crucial as well.

Any Software Performance Test Plan should have the minimum contents as
mentioned below,

• Performance Test Strategy and scope definitions.

• Test process and methodologies to follow in different test

phases.

• Test tool details (Tool Evaluation, Selection, Configuration

and Add-ins)

• Test cases details including scripting, library development

and script maintenance mechanisms to be used in every performance Test
Phase.

• Resource allocations and responsibilities for Testers.

• Risk management definitions.

• Test Start /Stop criteria along with Pass/Fail criteria

definitions.

9
Load Runner
 • Test environment setup requirements. (Hardware Setup,
Software Setup, Os Configurations, Third Party Utilities Setup etc.)

• Virtual Users, Load (Iterations Vs Users), Volume Load

Definitions for Different Load/Performance Test Phases.

• Results Analysis and Reporting format definitions.

Load / Performance Tool Evaluation & Selection

Evaluation of tool is an important task in performance test automation where

there are several parameters to be considered. In case the tool evaluation and
selection is completely done by the client then we don’t need to consider this.
While selecting any tool for Load or/and Performance testing the following things
should be analyzed:

• Test budget Vs Available Load /Performance tools in the market mapping

to the Budget.

• Protocols, Development & Deployment Technologies, Platforms,

Middle-Ware tools / Servers, Third-Party Integrations with the application
under test Vs support for these factors in the available tools with
prioritization of the availability in the tool for the scope of expected test.

• Complexity of the tool usage Vs Availability of the tool experts along

with the timeline requirements for the tool scripting / load scenario
creation / tool configuration with respect to man-hours and other resource
requirements.

• Tool limitations and work-around factors mapped with the current

scope of testing.

• Tool’s integration / Portability factors with other tools used for

monitoring and Analyzing.

• On evaluation and selection of the base tool, third party monitors /

tools to be used in integration with main Load testing tool should be
defined.

Test Process / Methodology and Test Strategy

10
Load Runner
 Test process, methodology and strategy for any Performance testing will
definitely vary from project to project based on the client’s requirement /
company. By using a methodology a project can make sure that resources are
applied to the problem effectively and the people involved in the work in a
structured way.

Performance Test Start/Stop Criteria

In performance testing, there should be a defined Start and Stop criteria to avoid
the ad-hoc process resulting in weird outputs. This Start / Stop criteria should be
well defined in the performance Test Plan along with all the risk management
factors.
The start of any Performance testing should be done for any application or any
module of the application during the design, database structure and application’s
development platform is finalized with no major changes at any layers of the
application that effect directly or indirectly.
Once the Performance test is conducted, results are collected and analyzed. The
cycles of tuning will be carried out but definitely the end criteria should be well
defined to stop this Performance testing. Tuning will be done based on various
factors like product release timelines (Project Deadlines) and client requirements
in terms of system stability, scalability and reliability under load and performance
tolerance criterions.
In some projects the end criteria is defined based on the client performance
requirements defined for each section of the application. So, if the product
reaches to the expected level then that can be considered as the end criteria for
performance testing.

Test Environmental Setup and Pre-Requisites

In performance testing, test environment is composed of hardware environment
like Machine, Memory, Virtual Memory, Number of CPU’s etc., software
environment like OS, File system settings, free space for logs and application
configuration along with middle tier server and associated configurations,
Database, Network etc. his setup also involves the Automation Tools installation,

11
Load Runner
 configurations, Agent Installations, setup in Load Clients, Third Party Monitors on
Clients / Servers etc.

Analysis and Report Generation

Once the results are ready, analysis is the most important job in the whole
performance test. The main criteria for analysis should be to find out the
bottlenecks and to track out the transactions / users / hits that create the highest
level of performance reduction.

The main factors to be considered are:

• Request / Response Timings from client to server
• Transaction Timings Vs Users
• Transaction Timings Vs Load
• Transaction Timings Vs CPU Utilization
• Server Memory Utilization
• Database server performance using DB server monitors
• Web / Application server performance
• Paging Rate Ratios
• Load Balancer Factors

After consolidating all the results, the statistical analysis should be done and the
performance test report should be created such that the development, design-
architecture, database group can get a clear idea about the application’s
performance and all the other factors affecting it. After analysis, based on the
scope tuning like OS tuning, Database Tuning, Memory Up-gradations, CPU
counts in Multi-CPU machine, Code tuning, Application Server cache tuning or
other tunings can be done. Same tests executed earlier should be repeated in the
tuned environment to make sure that there are differences in the results before
and after tuning. These tuning cycles can be repeated until the required
performance is achieved.

Pass / Fail / Exit Criteria

At the end of the Performance test execution, we can find the number of failed
users, failed transactions or failed scripts. But the Pass / Fail criteria for a
particular test should be well defined before execution of the test, as this

12
Load Runner
 criterion entirely depends on the scope of testing and the environment. The failed
items may be due to any of the performance test parameters. So the failed item
details should be included in the application’s limitations list. By doing so, the
limitations list can give a clear picture about the performance limitations with
respect to load parameters which in turn will give inputs for the tuning /
enhancements for the application. Based on the test results, tuning cycles will be
repeated but the exit criteria should be pre-defined to avoid unnecessary tuning
life cycles.

Common Implementation Issues

This section describes the most common issues that occur during the load testing
and also suggests strategies for addressing those issues.

Preparation

Planning:
The biggest issue with planning is that it is not performed adequately. A good
process is to allow three months for a load test with completion at least a month
before “go live”. Preparation, development, execution/analysis and results
summary each should be allocated one month. Typically the results summary can
be performed in parallel with late-term project tasks.

Analysis and Design:

The first issue to be considered is “what should I test?” and the related question
“what activity does my user community create”. In many ways this is black art of
testing. Often the vendors, system architects, IT and business managers make
educated guesses about the “representative user load profile”. This process
involves a study of the system’s operation profile. This also includes an interview-
analyze-negotiate process with user community.

Business processes:
High volume business processes / transactions should be built into the test.
Choosing too few might leave gaps in the test while choosing many will expand
the script creation time. We can effectively model the most common 80 percent
of the transaction throughput but trying to achieve greater accuracy is difficult
and expensive.

13
Load Runner
Test Environment Setup

Load testing requires a complete, stable and independent environment for testing.

Hardware:
To get the reliable results, the physical components of the test environment must
represent the production environment. This offers the advantage for ongoing
testing. This is particularly useful for maintenance or system expansion.

Software:
In addition to the hardware required for load test, the test bed must also have
fully installed and functioning software. Since Load Runner (Vuser) represents a
real time user (s), the system should successfully support all the user actions.

Network:
Organization’s network infrastructure shared by computers may create significant
amount of “background noise”. While it is probably impossible to accurately
model each and every network access, but it is good to examine the current
network utilization and understand the impact on network traffic.

Geography:
Often the system under test will support the global enterprise. In this
environment tests may often need to be run at remote sites across the WAN.
WAN connectivity need to be emulated in the lab or assumptions must be made.

Customization

Customization of Load Runner is possible and has been performed in many

environments. It will take considerable engineering effort and is not an out-of-
box-solution. Specifically customization of load runner requires:
1. Load Runner expert familiar with Template programming
2. System specialist familiar with the application’s API and administration.
3. An additional one-month of project time to investigate and construct a custom
Load Runner drive.

Functional Support:
One of the most important factors in script creation productivity is the amount of
functional support provided – access to individuals who understand the

14
Load Runner
 application’s functionality. When a test team member encounters a functional
error while scripting, the business Operation doesn’t function properly. The team
member typically has to stop since he or she is not equipped with the skill to
solve the issue. At that point, script creation is temporarily halted until a
functional team member helps to resolve the issue.

Test design Quality:

The second factor of script development is the quality of the test design. Ideally
the test design should specify enough information for an individual with little or
no experience to test builds. System test documentation is an excellent source of
this information. Often designs are incorrect or incomplete as a result, any
omission will require functional support to complete the script development.

Business Process Stability:

To load/ stress a large system, at first the system’s business processes should
function properly. It is typically not effective to attempt a load test a system that
doesn’t even work for the user. This means that the system needs to be nearly
completed.

System Changes:
The last factor in script development is the frequency of system changes. For
each system revision, test scripts must be evaluated. Each test should be
unaffected, require simple rework, or complete reconstruction. While testing tools
are used to minimize the effect of the system change, which will reduce the
scripting time.

Test Data Load:

The system should be loaded with development test data. This data often comes
from a legacy conversion and predecessor to the volume data for the test.

Execution/Analysis

System Changes:
Often there is shift between the systems used for development and execution
analysis. The first step is to execute all the business scenarios with single user.

15
Load Runner
 This typically involves working with each test to account for system changes like
difference in data load, configuration, system version etc which may cause
issues. It is recommended to perform a backup of the development environment
and/or database that may be restored for execution.

Volume data:
System performance may vary widely with the volume of the system data. The
system under test should have sufficient data to model the size of the product
database at the time of deployment. It is also good to expand the database size
to model the system performance with more than expected data for one or two
years.

System Support:
Just as the development requires functional support, execution/analysis requires
system support because the test team may not understand the system
architecture and technology. The purpose of the system support is to help in
interpreting the load runner results and performance log files. While Load Runner
will describe the facts based on which the system support will describe the reason
and suggest remedies for the problems. These suggestions can be implemented
and the tests will be rerun to cross verify the new implementation.

Introduction to Load Runner

Mercury’s Load Runner is an efficient & effective tool used to conduct

performance testing. The following reasons prove its efficiency:

1. On - demand Production Work Loads: Load Runner is able to drive ‘n’

number of virtual users executing business process to emulate the real time
conditions.

2. Enterprise Environment Support: Load Runner supports nearly 40

protocols; this includes Web, J2EE, .NET, XML, SAP, Siebel, Oracle,
PeopleSoft, wireless, Citrix, and other client-server applications.

16
Load Runner
 3. Enterprise Monitoring Support: Load Runner’s real-time performance
monitors provide detailed metrics on all parts of the system under test. This
includes Web servers, application servers, databases, ERP and CRM systems,
firewalls, and load balancers. Load Runner can identify hardware limitations
and software configuration issues that might otherwise go undetected.

4. Diagnostics: Load Runner can be used to trace, time and troubleshoots

individual application components under load. Users can now drill down from
a slow end-user transaction to the bottlenecked method or SQL statement
that is causing the slowdown. This granularity of results data ensures that
every load test provides development with actionable results, thus reducing
the cost and time required to optimize J2EE, Siebel, and Oracle deployments.

5. Infrastructure Tuning: Load Runner will provide a customizable

methodology to isolate and resolve infrastructure bottlenecks in the system
prior to going into production. Over 70 percent of performance bottlenecks
are due to system infrastructure issues. By detecting and resolving these in
pre-production, Load Runner saves valuable time and money.

6. Automatic Analysis: Load Runner’s Auto-Correlation wizard automatically

takes all the monitoring & diagnostics data and provides the top-five causes
of performance degradation. As a result, Load Runner has the industry’s
deepest and broadest analysis. Converting performance-testing results into
actionable, precise data to development dramatically reduces time to
resolution and allows for more testing cycles. This is a critical piece in
ensuring that a high-quality application goes into production.

7. Ease of Use: Load Runner is built from the ground up for QA users. It
provides a visual scripting language, Data and autocorrelation wizards and
Active Screen technology to make scripting and running of performance tests
as easy as possible.

Load Runner Terminology

Scenario: A scenario defines the events that occur during each testing session.
Scenario defines and controls the number of users to emulate, the actions that they
perform and the machines on which they run their emulations.

17
Load Runner
Vusers: In the scenario, Load Runner replaces human users with virtual users or
Vusers. When you run a scenario, Vusers emulate the actions of human users
working with your application.

Vuser Scripts: The actions that a Vuser performs during the scenario are described
in a Vuser script. When you run a scenario, each Vuser executes a Vuser script.

Transactions: To measure the performance of the server, you define transactions.

A transaction represents an action or a set of actions that you are interested in
measuring.

Controller: LoadRunner Controller will be used to manage and maintain your

scenarios. Using the Controller, you control all the Vusers in a scenario from a single
workstation.

Load generator: When you execute a scenario, the Load Runner Controller
distributes each Vuser in the scenario to a load generator. The load generator is the
machine that executes the Vuser script, enabling the Vuser to emulate the actions of
a human user.

Load Runner Vuser Technology

Remote agent
Dispatcher Vuser

Agent Vuser
Controller

Vuser

Load generator

18
Load Runner
Creating Scripts using Load Runner

We have to create the scripts by recording the steps specified in the business
scenario document. Creating the script consists of the flowing steps.

Record Vuser Script

Enhance Vuser Script

Configure Runtime
Settings

Run the Vuser Script in

Stand Alone Mode

Recording Vuser Scripts using VuGen

We can use VuGen to develop the Vuser script by recording user performing typical
business processes on a client application. Each Vuser script that you create with
VuGen can directly communicate with a server by executing calls to the server API
without relying on client software. In addition, when a user communicates directly
with the server, system resources are not used on a user interface. This lets you run
large number of Vusers on a single workstation.

1. Open the Virtual User Generator.

2. Select the Protocol for recording.

3. Click on Start recording.

4. Enter URL/Path of the application for recording.

19
Load Runner
 5. Record all the steps for the specified business scenario

Enhancing the scripts

While recording a script or after recording, you can enhance its capabilities by adding
the following functions to the script.

1. General functions

2. Protocol specific functions.

3. Standard ANSI C functions.

4. Parameterization.

5. Correlation.

General Vuser functions:

General Vuser functions greatly enhance the script functionality of any Vuser script.
You can use general Vuser functions to measure the server performance, control
server load and debugging code or retrieve run-time information about the Vusers
participating in the scenario.

Protocol Specific Functions:

There are several library functions that you can use to enhance your script. For
example LRS functions for Win Socket applications, LRD functions for database
applications etc.

Standard ANSI C Functions:

You can enhance your Vuser scripts by adding ANSI C functions to the script. ANSI C
functions allow you to add comments, Control flow statements and conditional
statements so on to Vuser scripts.

Parameterization:

20
Load Runner
When you record a business process VuGen generates a script that contains actual
values used for recording. Suppose you want to perform script actions like query,
Submit using different values from those recorded values. To do this you need to
replace the recorded values with parameters. This is called as parameterization.

Correlation:

Correlation is used to capture the dynamic value for a particular field at run time.

The primary reason for correlating statement is

1. To simplify or optimize your code

2. For Dynamic Data

3. To accommodate unique records.

You can also enhance the scripts by

1. Inserting transactions into a Vuser script.

2. Inserting Rendezvous point into a Vuser script

3. Inserting comments into the script.

4. Sending messages output.

5. Handling errors in Vuser Scripts.

Configuring Run time Settings

The run time settings will define the way that the script runs. These settings are
stored in the file default.cfg located in the Vuser directory. Runtime settings are
applied to Vusers when you run the scripts in VuGen or controller. We have to adjust
the run time settings based on the no. of transactions/hour. We will be calculating
this using the formula.

Transactions / hour = no. of user* 60 /

21
Load Runner
Action section Execution time + Think time + Pacing Time)

Run Logic

Number of Iterations: Based on the number of iterations set, Load Runner repeats
all the Actions for specified number of times. If you specify scenario duration in the
Controller’s Scheduling settings, the duration setting overrides the Vuser iteration
settings. This means that if the duration is set to one hour, the Vusers will continue
to run as many iterations as possible in one hour, even if the run-time settings set to
one iteration.

Pacing

The Pacing Run-Time settings let you control the time between iterations. The pace
tells the Vuser how long to wait between iterations of your actions.

22
Load Runner
You instruct the Vusers to start each iteration using one of the following

As soon as the previous iteration ends: The new iteration begins as soon as the
previous iteration ends.

After the previous iteration ends with a fixed or random delay of …: Starts
each new iteration, with a delay of specified amount of time after the end of the
previous iteration. Specify either an exact number of seconds or a range of time.

At fixed or random intervals, every … [to …] seconds: You can specify the time
between iteration either a fixed number of seconds / range of seconds. For example,
you can specify to begin a new iteration every 30 seconds, or at a random rate
ranging from 30 to 45 seconds from the beginning of the previous iteration. Each
scheduled iteration will only begin when the previous iteration is complete. The

23
Load Runner
actual amount of time that the Vuser waits between the end of one iteration and the
start of the next one appears in the Execution Log when you run the script.

Log

Enable Logging
This option enables automatic logging during replay. VuGen writes log messages that
you can view in the Execution log. This option only affects automatic logging and log
messages issued through lr_log_message. Messages sent manually, using
lr_message, lr_output_message, and lr_error_message, are still issued. The
Log run-time settings allow you to adjust the logging level depending on your
development stage.

Standard Log
Creates a standard log of functions and messages sent during script execution to use
for debugging. Disable this option for large load testing scenarios. If the logging level

24
Load Runner
is set to Standard, the logging mode is automatically set to JIT logging when adding
it to a scenario.

Extended Log
Creates an extended log, including warnings and other messages. Disable this option
for large load testing scenarios. If logging is disabled or if the level is set to
Extended, adding it to a scenario does not affect the log settings. You can specify
additional information, which should be added to the extended log using the
Extended log options:

Parameter substitution

Select this option to log all parameters assigned to the script along with their values

Data returned by server

Select this option to log all of the data returned by the server.

Advanced trace

Select this option to log all of the functions and messages sent by the Vuser during
the session. This option is useful when you debug a Vuser script.

Think Time

25
Load Runner
Vuser think time emulates the time that a real user waits between actions. For
example, when a user receives data from a server, the user may wait several
seconds to review the data before responding. This delay is known as the think time.
VuGen uses lr_think_time functions to record think time values into your Vuser
scripts.

By default, when you run a Vuser script, the Vuser uses the think time values that
were recorded into the script during the recording session. VuGen allows you to use
the recorded think time, ignore it, or use a value related to the recorded time.

Ignore think time: Ignore the recorded think time option replay the script by
ignoring all lr_think_time functions.

Replay the think time: The second set of think time options let you use the
recorded think time.

26
Load Runner
 As recorded: During replay, use the argument that appears in the
lr_think_time function. For example, lr_think_time (10) waits ten seconds.

Multiply recorded think time by: During replay, use a multiple of the
recorded think time. This can increase or decrease the think time applied
during playback

Use random percentage of the recorded think time: Use a random

percentage of the recorded think time. You set a range for the think time
value by specifying a range for the think time.

Limit think time to: Limit the think time’s maximum value to the specified
value.

Miscellaneous Run-Time Settings

Error Handling
You can specify how a Vuser handles errors during script execution. By default, when
a Vuser detects an error, it exits. You can use the run-time settings to instruct a
Vuser to continue script execution when an error occurs. To do so, select the
Continue on Error check box in the Miscellaneous run-time settings.
You can also instruct VuGen to mark all transactions in which an lr_error_message
function was issued, as Failed. The lr_error_message function is issued through a
programmed If statement, when a certain condition is met.

27
Load Runner
Multithreading
Vusers support multithread environments. The primary advantage of a multithread
environment is the ability to run more Vusers per load generator. Only thread safe
protocols should be run as threads.

• To enable multithreading, click Run Vuser as a thread.

• To disable multithreading and run each Vuser as a separate process, click

Run Vuser as a process.

Automatic Transactions
You can instruct Load Runner to handle every step or action in a Vuser script as a
transaction. This is called using automatic transactions. Load Runner assigns the step
or action name as the name of the transaction. By default, automatic transactions
per action are enabled.

• To disable automatic transactions per action, clear the Define each action
as a transaction check box. (enabled by default)

28
Load Runner
 • To enable automatic transactions per step, check the Define each step as a
transaction check box. (disabled by default)

Speed Simulation

Using the Speed Simulation settings, you can select a bandwidth that best emulates
the environment under test. The following options are available:

Use maximum bandwidth: By default, bandwidth emulation is disabled and

the Vusers run at the maximum bandwidth that is available over the network.

Use bandwidth: Indicate a specific bandwidth level for your Vuser to

emulate. You can select a speed ranging from 14.4 to 512 Kbps, emulating
analog modems, ISDN, or DSL.

Use custom bandwidth: Indicate a bandwidth limit for your Vuser to

emulate. Specify the bandwidth in bits, where 1 Kilobit=1024 bits.

Proxy

29
Load Runner
Proxy runtime settings are used to provide the Proxy server details to the scripts.

No Proxy: For all the Vusers make the connection directly to the internet, without
using Proxy Server.

Obtain the Proxy Server settings from the default Browser: All Vusers use the
proxy settings of the browser on the machine they are running. This is the default
option.

User Custom Proxy: All Vusers connect to the Internet through a Proxy using the
settings you specify.

Use Proxy: Enabling this check box provides the Address of the proxy to use and
port. If the proxy requires authentication information provide this in the proxy
authentication window specified below.

30
Load Runner
Executing Scenario Using Controller

Scenario: A scenario defines the events that occur during each testing session.
Scenario defines and controls the number of users to emulate, the actions that they
perform and the machines on which they run their emulations.
Manual Scenario: Select this method if you want to build a manual scenario. You
build a manual scenario by creating groups and specifying the script, the load
generator, and the number of Vusers included in each group.
Use the Percentage Mode to distribute the Vusers among the scripts:
Select this option if you want to build a manual scenario by specifying a
number of Vusers to be distributed among the selected Vuser scripts.
Goal Oriented Scenario: Select this method to have Load Runner build a scenario
for you. In a goal-oriented scenario, you define the goals you want your test to
achieve, and Load Runner automatically builds a scenario for you, based on these
goals. This scenario is mainly used for stress testing.

Scheduling a Scenario

Ramp up: Specifies the entry criteria for the users into the server.
Load All users Simultaneously: Will load all the users at a time after the
scenario is started.
Start {No. of users} after every {Time}: will load the specified number of
users in to the server in a given time.

Duration:
Run until Completion: Will run the scripts once. If the number of iterations
specified inside the Runtime Settings, Script will be executed for specified
number of iterations for each user.

31
Load Runner
 Run for {Time} after ramp up has been completed: Scripts will be run
for specified duration after all the users were ramped up. Script will be
repeated for ‘n’ number of times based on the duration specified. On selecting
this option, Run Logic runtime setting specified in the script will be
overwritten.

Run Indefinitely: Scripts will be running indefinitely till we stop the

scenario. This option will be used for conducting the Volume testing of an
application.

Ramp Down: Specifies the exit criteria for the users from the server.

Stop all Vusers Simultaneously: Exits all the users from the server at a
time.

Stop {No. of users} after every {Time}: Stops a certain number of

Vusers within a specified time frame.

Setting Up Load Generator Machines

After installing the load runner agent software on the Load Generator machine we
have to configure the Agent for connectivity with the controller machine. Follow the
below steps for configuring the Agent
1. Go to Programs -> Load Runner ->Advance Settings -> Agent Configuration. On
selecting this it will display the following window

Select the Enable Firewall Agent option and click on Settings button. It will
display all the setting related to the agent configuration. Set the following properties
for configuring the agent.

Local Machine Key: Specify the IP Address of the controller machine to establish a
unique connection between the Controller the agent machine.

32
Load Runner
Configuring Load Generators

For configuring the load Generator click on the Generator button in Controller
design window and add the load generator machine by specifying the name of the
machine. Click on Connect button to check whether the connectivity between the
controller machine and Load generator machine can be established. If the
connectivity is failed please check the following.
1. Login into the Load Generator machine ensure that Agent process is launched
and running.
2. Check the network connectivity between controller machines by running the
command Ping machine_name at command prompt of the controller/ agent
machine.
3. Verify the Load Generator Machine name / IP address in the load generator
window.
4. Verify whether the agent machine configured using Agent configuration.
5. Verify whether any firewall exists.

Setting up the Monitors

Monitors

Load Runner uses a suite of integrated performance monitors to quickly isolate

system bottlenecks with minimal impact on the system. The suite consists of
monitors for the network, application servers, Web servers and database servers.
These monitors are designed to accurately measure the performance of every single
tier, server and component of the system during the load test. By correlating this
performance data with end-user loads and response times, IT groups can easily
determine the source of bottlenecks. In addition, they can collect and manage all

system performance data from the Load Runner Controller. We can configure the
monitors in controller to capture these details.

Configuring Load Runner Monitors.

1. Go to Run tab of controller window.

2. Select the corresponding monitor (Web / Application / Database server) from
Available Graphs section

33
Load Runner
 3. Drag and drop the monitor into any of the graphs.
4. Right click on the graph and select Add Measurements

Now it will display the following window.

5. Click on Add(top) and enter the machine name of the Web/App/DB server
6. Now click on the Add (Down) button and select the required Performance
counters to be measured during the scenario execution.
7. Click on OK button.

If controller is not able to capture the details after configuring please ensure the
following.

1. Verify the connectivity between the Load Runner Controller machine and
Server Machine.
2. Verify whether you are Admin user or you have sufficient privileges to capture
the performance counters from the server machine.

34
Load Runner
Analyzing Results

Once the results are ready, analysis is the most important job in the whole
performance test if the requirements of performance test are not defined. The main
criteria for analysis should be to find out the bottlenecks and we should be able to
track the transactions / users / hits that create the highest level of performance
reduction. Based on the test run, Load Runner will generate various graphs and
reports. This phase contains:

Analysis

Once the load test is completed user can correlate the various graphs to analyze the
results and to identify the business scenarios causing high CPU utilization and
network traffic.

Monitoring

It includes observation of what is happening in each tier of the application, web

server, Application server and Database server. We can analyze the system
resources utilized and network delays.

Tuning

It includes observation and understanding of the particular query or activity which

are not meeting the response time and tuning it by creating indexes, or reordering
the queries and by modifying the hardware( if required).

Diagnostics

Performance testers have a single, unified view of how individual tiers, components
and SQL statements impact the overall performance of a business process under load
conditions. With diagnostics, performance testers using Load Runner can see all
components touched by an end – user transaction. More over, users can also see

how long each component takes and how many times it is called. With this
information corresponding team can focus on resources to improve the end-user
experience by targeting the most significant area of the web application and
database server bottlenecks.

35
Load Runner
Summary Report

The summary Report provides the information about the scenario execution. It
provides the information like Running Vusers, Throughput, Hits Per second, HTTP
Responses per Second, Transaction Summary and Average Transaction Response
time. 90 percent column indicates the maximum response time for ninety percent of
the Transactions.

36
Load Runner
Running Vusers Graph

Running Vusers graph displays the number of users that executed the scripts and
their status during the execution. This graph is useful to determine the load on the
server at any given moment.

Hits per Second

The Hits per Second graph shows the number of HTTP requests made by Vusers to
the Web servers during the scenario execution. This graph helps you to evaluate the
amount of load Vusers generate in terms of number of hits.

37
Load Runner
 38
Load Runner
Transaction Summary

Transaction summary graph summarizes the number of transactions in the scenario

that failed, Passed, stopped and ended with errors.

39
Load Runner
Average Transaction Response Time

This graph displays the average time taken to perform transactions during each
second of the scenario run.

40
Load Runner
Tuning

Information Captured for Tuning

For analyzing the results and for identifying the bottlenecks capture the following
information during the Scenario execution.
1. Run the SQL Server Trace file during the Scenario Run.
2. Capture the Performance counters during the run into the Load Runner
controller (if you have access to LR Monitors) otherwise use the windows
performance monitor to capture the same.
3. Run the Diagnosis tools, if you find any Hardware issues.

Performance Counters to be captured

1. CPU Utilization
2. Avg. Disk Queue Length
3. Network Interface
4. System
5. Server
6. Process
7. Physical Disk
8. Logical Disk
9. Memory

Few Tips on Resources utilization of the Servers

Parameter Tips
None of the CPUs should have utilization of more than
Utilization 80 percent.
Avg. Disk Queue Length Should not be more than 2.
Load Average More than three processes should not wait for the CPU.
Any non-DB, system processes utilizing CPU intensively
CPU Intensive Processes warrant attention.
Swapping Minimal to no swapped-out processes
Paging Minimize page faults
Data Buffer Hit Ratio More than 70 percent
Disk should have less than 50-70 I/Os per second. Also,
High Disk Activity the disks should be less than 60 percent busy.

41
Load Runner
 Minimal to no disk request queue. A queue of 2-4
Long Disk Queues warrants attention.

Un-indexed Tables All the tables should have at least one Primary Key.

Measurement Calculations:
The less time per transaction, the greater a system's throughput, as expressed by
the formula derived from Little's Law:
Server Thruput = # of real simultaneous users / ( Average Response Time + Think
Time )

So when no think time is used, the number of virtual users can be calculated by this
formula suggested by Menasce & Almeida:

# of real simultaneous users * [ Response Time / ( Response Time + Think Time ) ]

For example, to simulate 1,000 real users using 2 second response time and 20
second think times:

1,000 x [ 2 / ( 2 + 20 ) ] = 91

System Load

is measured by a rate -- the total number of tasks performed during a certain

period (an hour, day, week, month, quarter, year, etc.). Example: 300 transactions
per second.

There can be several levels of load:

o Peak number per hour or per second is typically used for system
sizing.
o Normal load levels are used to plan on-going support requirements.
o Low load levels are used to plan the percentage of resources which
can be taken off-line for repairs and upgrades.

42
Load Runner
Measures of Variation

The average measures the central tendency in a group of numbers. For

example, the average of 10, 20, and 30 is 20. This is calculated by adding all
three numbers together (10 + 20 + 30 = 60) and dividing this sum by the
number of items added together (in this example, three).

Standard Deviation

Standard deviation is a statistical measure of the extent of variation in a set

of data. The standard deviation tells you the extent of the spread of
individual items around the average.

The number in the 90% column is shown after a run is completed in the
Analysis program, not in the Controller) because it is calculated by sorting
all items by value obtained during the run, then identifying the value
associated the item at the top 10% of all items.

References
1. http://www.aptest.com/resources.html
2. http://www.sql-server-performance.com
3. http://www.mercuryinteractive.com/
4. http://www.qaforums.com

43
Load Runner