Performance Testing Using Load Runner

Load Runner

1

TABLE OF CONTENTS Introduction • Definition for Performance testing Types of Performance Testing Need for Performance testing Automate Performance Testing Automated Performance Testing Phases 5 5 5 5 6 6 7 7 7 8 8 10 10 11 11 12 12 12 13 13 13 14 15 16 17 17 18 18 19

• •

• • • • • • • • • • • • • • • • • • •

Requirements collection Design Build Execute Analysis and Tuning

Performance Test Planning Load / Performance Tool Evaluation & Selection Test Process / Methodology and Test Strategy Performance Test Start/Stop Criteria Test Environmental Setup and Pre-Requisites Analysis and Report Generation Pass / Fail / Exit Criteria

Common Implementation Issues Preparation Test Environment Setup Customization Execution/Analysis

Introduction to Load Runner Load Runner Terminology Load Runner Vuser Technology

Creating Scripts Using Load Runner Recording Vuser Scripts using VuGen Enhancing the scripts

Load Runner

2

Configure Run time Settings

20

Executing Scenario using Controller

29 29 30 30 31

• • • •

Scheduling a Scenario Setting Up Load Generator Machines Configuring Load Generators Setting up the Monitors

Analyzing Results

33 34 35 36 37 38

• • • • •

Summary Report Running Vusers Graph Hits per Second Transaction Summary Average Transaction Response Time

Tuning

39 39 39

• •

Information Captured for Tuning Performance Counters to be captured

Load Runner

3

Abstract The incredible pace of change and the explosion of software complexity introduce tremendous risk into software development process. Rigorous testing is the most common strategy to quantify and reduce this risk to the business. The question for developers, QA teams, and management alike is how accurately and thoroughly they can validate the system’s performance before Go Live – without breaking the Budget. By conducting performance testing, an organization can verify that the application(s) meet the Business needs before Go Live. This ensures that there are no Deployment Risks, and Quantifies the impact of changes on End-User’s experience by pinpointing the Failed Components for Rapid Resolution. This paper presents a complete practical approach for performance testing and covers the best approach which includes the following: • • • • • • • • What is performance testing? Why should you automate it? What is the right process for performance testing? Who should be involved in a good performance test? What are the requirements need to be collected for performance Testing? How to plan for performance Testing? What are the common implementation issues faced during the performance testing? How to conduct a performance testing using Load Runner? o o o Creation of Scripts Enhancing Scripts Creation and Execution of Scenarios

Load Runner

4

o o

Analyzing Results Tuning the application.

Introduction What is Performance Testing? Performance Testing is aimed at verifying the system’s performance requirements like response time, Transactional throughput and no of concurrent users. Performance testing is used to accurately measure the End-to-End performance of a system prior to Go Live. It will identify the loop holes in Architectural Design which helps to tune the application. It includes the following: ♦ ♦ ♦ ♦ ♦ Emulating ‘n’ number of users interacting with the system using minimal hardware. Measuring End-User’s Response time. Repeating the load consistently Monitoring the system components under controlled load. Providing robust analysis and reporting engines.

Types of Performance Testing Load Testing: To verify that system can handle the expected load, on deploying it under Real Time conditions. Stress Testing: To identify the application’s Break Point, also to measures whether the application’s environment is configured to handle the expected or potentially unexpected high transactional volume. Scalability testing: Scalability testing integrates very well with performance testing. The purpose of scalability testing is to determine whether the application automatically scales to meet the growing user load.

Load Runner

5

Volume testing: To verify the stability of the system with respect to handling large amounts of data over extended period of time.

Need for Performance testing To verify the below mentioned:

 Application’s
 

response for the intended number of users

Application’s maximum load resisting capacity Application’s capacity of handling the number of transactions required by the Business Application’s stability under expected and unexpected user load Ensuring that users have proper response time on production

 

Automate Performance Testing Automating Performance Test is a discipline that leverages people, process, and technology to reduce the Risks of the Application. It is about applying production workloads to pre-deployment systems while simultaneously measuring system performance and End-User experience. Below are the loopholes in manual Load Testing that leads to Automation. • • • • • Difficult to measure the performance of the application accurately. Achieving the synchronization among the users. ‘n’ number of real time users are required to involve in Performance Testing Difficult to analyze and identify the results & bottlenecks. Increases the infrastructure cost.

Automated Performance Testing Phases

Planning/Design

Build

Execution Load Runner Analysis and Tuning 6

Requirements Collection Design This is the primary phase where team will be gathering the requirements of the performance testing. Requirements can be Business, Technical, System and Team requirements. Business requirements includes

1. An Application Overview:

Performance

Team

should

have

good

understanding of the Application prior to Testing

2. Business Processes List: Team should gather complete information about
list of key Business activities that will be performed by an End User on the system.

3. Business Process Flows: Document, which contains step-by-step details of
the Business processes identified by Business Team.

4. Transaction List: List of Key activities with in the Business process that
need to be performed in unit time. Technical requirements should be collected from the System Administrators and Database Administrators, which include the following: 1. Complete Architecture of the Environment 2. How many no. of users that the system should support during normal and peak hours. 3. How many no. of transactions per unit time should be processed by the system 4. Response time for all the Business Processes. 5. What are the different system workloads that will be experienced in production

Load Runner

7

6. How the load balancing will be done during the testing

Build This phase consists of automating the requirements collected during the design phase. It includes

1. Scripting: Record the business processes gathered in the design phase into
Automated scripts.

2. Transactions:

These are the points that measure the performance of a

particular key activity with in the business process.

3. Parameterization: Input data needs to be passed for the scripts for
performing the Business activity on an application.

4. Scenarios: The complete End-to-End Business Workflow, which simulates the
Real Time Environment.

5. Monitors: Decide which servers or machines need to be monitored.
Environment setup consists of assembling Hardware, Software and Data required for executing a successful real load. This may involve working with the systems, DBA, Operations and Business Teams. Execute Execution will be done in multiple phases. It consists of various types of testing like Baseline Testing This testing is done to ensure that both applications’ functionality & environment setup for performance testing is proper. This will be done with 10 to 20 users. Performance Tests This includes running the scripts with Average and Maximum load with specified no of transactions. We will be simulating the real time environment with different load conditions using different Scenarios, Think Time, Pace Time and no. of users. Benchmark Tests These are designed to measure and compare the performance of each machine type, environment or build of the application in ideal situation. These tests are run after

Load Runner

8

the system undergoes scalability testing to understand the performance impact of different architectures.

Analysis and Tuning During the performance testing we will be capturing all the details related to the system like Response time and System Resources for identifying the major bottlenecks of the system. After the bottlenecks are identified we have to tune the system to improve the overall performance. Performance Optimization This quantifies to improve the performance of the system as measured by improved end-user response time or by reducing the overall required hardware infrastructure. Performance Test Planning To make any operation successful, planning plays a major role. According to the 80-20 theories, 80% of the time should be spent in planning and 20% in real time plan execution/implementation. In the similar procedure, software performance test planning is very crucial as well. Any Software Performance Test Plan should have the minimum contents as mentioned below, • • phases. • and Add-ins) • Phase. • • • definitions. Resource allocations and responsibilities for Testers. Risk management definitions. Test Start /Stop criteria along with Pass/Fail criteria Test cases details including scripting, library development Test tool details (Tool Evaluation, Selection, Configuration Performance Test Strategy and scope definitions. Test process and methodologies to follow in different test

and script maintenance mechanisms to be used in every performance Test

Load Runner

9

Test environment setup requirements. (Hardware Setup,

Software Setup, Os Configurations, Third Party Utilities Setup etc.) • Virtual Users, Load (Iterations Vs Users), Volume Load

Definitions for Different Load/Performance Test Phases. • Results Analysis and Reporting format definitions.

Load / Performance Tool Evaluation & Selection Evaluation of tool is an important task in performance test automation where there are several parameters to be considered. In case the tool evaluation and selection is completely done by the client then we don’t need to consider this. While selecting any tool for Load or/and Performance testing the following things should be analyzed: • Test budget Vs Available Load /Performance tools in the market mapping to the Budget. • Protocols, Development & Deployment Technologies, Platforms,

Middle-Ware tools / Servers, Third-Party Integrations with the application under test Vs support for these factors in the available tools with prioritization of the availability in the tool for the scope of expected test. • Complexity of the tool usage Vs Availability of the tool experts along

with the timeline requirements for the tool scripting / load scenario creation / tool configuration with respect to man-hours and other resource requirements. • Tool limitations and work-around factors mapped with the current

scope of testing. • Tool’s integration / Portability factors with other tools used for

monitoring and Analyzing. • On evaluation and selection of the base tool, third party monitors /

tools to be used in integration with main Load testing tool should be defined. Test Process / Methodology and Test Strategy

Load Runner

10

Test process, methodology and strategy for any Performance testing will definitely vary from project to project based on the client’s requirement / company. By using a methodology a project can make sure that resources are applied to the problem effectively and the people involved in the work in a structured way.

Performance Test Start/Stop Criteria In performance testing, there should be a defined Start and Stop criteria to avoid the ad-hoc process resulting in weird outputs. This Start / Stop criteria should be well defined in the performance Test Plan along with all the risk management factors. The start of any Performance testing should be done for any application or any module of the application during the design, database structure and application’s development platform is finalized with no major changes at any layers of the application that effect directly or indirectly. Once the Performance test is conducted, results are collected and analyzed. The cycles of tuning will be carried out but definitely the end criteria should be well defined to stop this Performance testing. Tuning will be done based on various factors like product release timelines (Project Deadlines) and client requirements in terms of system stability, scalability and reliability under load and performance tolerance criterions. In some projects the end criteria is defined based on the client performance requirements defined for each section of the application. So, if the product reaches to the expected level then that can be considered as the end criteria for performance testing. Test Environmental Setup and Pre-Requisites In performance testing, test environment is composed of hardware environment like Machine, Memory, Virtual Memory, Number of CPU’s etc., software environment like OS, File system settings, free space for logs and application configuration along with middle tier server and associated configurations, Database, Network etc. his setup also involves the Automation Tools installation,

Load Runner

11

configurations, Agent Installations, setup in Load Clients, Third Party Monitors on Clients / Servers etc. Analysis and Report Generation Once the results are ready, analysis is the most important job in the whole performance test. The main criteria for analysis should be to find out the bottlenecks and to track out the transactions / users / hits that create the highest level of performance reduction.

The main factors to be considered are: • • • • • • • • • Request / Response Timings from client to server Transaction Timings Vs Users Transaction Timings Vs Load Transaction Timings Vs CPU Utilization Server Memory Utilization Database server performance using DB server monitors Web / Application server performance Paging Rate Ratios Load Balancer Factors

After consolidating all the results, the statistical analysis should be done and the performance test report should be created such that the development, designarchitecture, database group can get a clear idea about the application’s performance and all the other factors affecting it. After analysis, based on the scope tuning like OS tuning, Database Tuning, Memory Up-gradations, CPU counts in Multi-CPU machine, Code tuning, Application Server cache tuning or other tunings can be done. Same tests executed earlier should be repeated in the tuned environment to make sure that there are differences in the results before and after tuning. These tuning cycles can be repeated until the required performance is achieved. Pass / Fail / Exit Criteria At the end of the Performance test execution, we can find the number of failed users, failed transactions or failed scripts. But the Pass / Fail criteria for a particular test should be well defined before execution of the test, as this

Load Runner

12

criterion entirely depends on the scope of testing and the environment. The failed items may be due to any of the performance test parameters. So the failed item details should be included in the application’s limitations list. By doing so, the limitations list can give a clear picture about the performance limitations with respect to load parameters which in turn will give inputs for the tuning / enhancements for the application. Based on the test results, tuning cycles will be repeated but the exit criteria should be pre-defined to avoid unnecessary tuning life cycles.

Common Implementation Issues This section describes the most common issues that occur during the load testing and also suggests strategies for addressing those issues. Preparation Planning: The biggest issue with planning is that it is not performed adequately. A good process is to allow three months for a load test with completion at least a month before “go live”. Preparation, development, execution/analysis and results summary each should be allocated one month. Typically the results summary can be performed in parallel with late-term project tasks. Analysis and Design: The first issue to be considered is “what should I test?” and the related question “what activity does my user community create”. In many ways this is black art of testing. Often the vendors, system architects, IT and business managers make educated guesses about the “representative user load profile”. This process involves a study of the system’s operation profile. This also includes an interviewanalyze-negotiate process with user community. Business processes: High volume business processes / transactions should be built into the test. Choosing too few might leave gaps in the test while choosing many will expand the script creation time. We can effectively model the most common 80 percent of the transaction throughput but trying to achieve greater accuracy is difficult and expensive.

Load Runner

13

Test Environment Setup Load testing requires a complete, stable and independent environment for testing. Hardware: To get the reliable results, the physical components of the test environment must represent the production environment. This offers the advantage for ongoing testing. This is particularly useful for maintenance or system expansion. Software: In addition to the hardware required for load test, the test bed must also have fully installed and functioning software. Since Load Runner (Vuser) represents a real time user (s), the system should successfully support all the user actions. Network: Organization’s network infrastructure shared by computers may create significant amount of “background noise”. While it is probably impossible to accurately model each and every network access, but it is good to examine the current network utilization and understand the impact on network traffic. Geography: Often the system under test will support the global enterprise. In this environment tests may often need to be run at remote sites across the WAN. WAN connectivity need to be emulated in the lab or assumptions must be made. Customization Customization of Load Runner is possible and has been performed in many environments. It will take considerable engineering effort and is not an out-ofbox-solution. Specifically customization of load runner requires: 1. Load Runner expert familiar with Template programming 2. System specialist familiar with the application’s API and administration. 3. An additional one-month of project time to investigate and construct a custom Load Runner drive. Functional Support: One of the most important factors in script creation productivity is the amount of functional support provided – access to individuals who understand the

Load Runner

14

application’s functionality. When a test team member encounters a functional error while scripting, the business Operation doesn’t function properly. The team member typically has to stop since he or she is not equipped with the skill to solve the issue. At that point, script creation is temporarily halted until a functional team member helps to resolve the issue.

Test design Quality: The second factor of script development is the quality of the test design. Ideally the test design should specify enough information for an individual with little or no experience to test builds. System test documentation is an excellent source of this information. Often designs are incorrect or incomplete as a result, any omission will require functional support to complete the script development. Business Process Stability: To load/ stress a large system, at first the system’s business processes should function properly. It is typically not effective to attempt a load test a system that doesn’t even work for the user. This means that the system needs to be nearly completed. System Changes: The last factor in script development is the frequency of system changes. For each system revision, test scripts must be evaluated. Each test should be unaffected, require simple rework, or complete reconstruction. While testing tools are used to minimize the effect of the system change, which will reduce the scripting time. Test Data Load: The system should be loaded with development test data. This data often comes from a legacy conversion and predecessor to the volume data for the test. Execution/Analysis System Changes: Often there is shift between the systems used for development and execution analysis. The first step is to execute all the business scenarios with single user.

Load Runner

15

This typically involves working with each test to account for system changes like difference in data load, configuration, system version etc which may cause issues. It is recommended to perform a backup of the development environment and/or database that may be restored for execution.

Volume data: System performance may vary widely with the volume of the system data. The system under test should have sufficient data to model the size of the product database at the time of deployment. It is also good to expand the database size to model the system performance with more than expected data for one or two years. System Support: Just as the development requires functional support, execution/analysis requires system support because the test team may not understand the system architecture and technology. The purpose of the system support is to help in interpreting the load runner results and performance log files. While Load Runner will describe the facts based on which the system support will describe the reason and suggest remedies for the problems. These suggestions can be implemented and the tests will be rerun to cross verify the new implementation. Introduction to Load Runner Mercury’s Load Runner is an efficient & effective tool used to conduct performance testing. The following reasons prove its efficiency:

1. On - demand Production Work Loads: Load Runner is able to drive ‘n’
number of virtual users executing business process to emulate the real time conditions.

2. Enterprise Environment Support: Load Runner supports nearly 40
protocols; this includes Web, J2EE, .NET, XML, SAP, Siebel, Oracle, PeopleSoft, wireless, Citrix, and other client-server applications.

Load Runner

16

3. Enterprise Monitoring Support: Load Runner’s real-time performance
monitors provide detailed metrics on all parts of the system under test. This includes Web servers, application servers, databases, ERP and CRM systems, firewalls, and load balancers. Load Runner can identify hardware limitations and software configuration issues that might otherwise go undetected.

4. Diagnostics: Load Runner can be used to trace, time and troubleshoots
individual application components under load. Users can now drill down from a slow end-user transaction to the bottlenecked method or SQL statement that is causing the slowdown. This granularity of results data ensures that every load test provides development with actionable results, thus reducing the cost and time required to optimize J2EE, Siebel, and Oracle deployments.

5. Infrastructure

Tuning:

Load

Runner

will

provide

a

customizable

methodology to isolate and resolve infrastructure bottlenecks in the system prior to going into production. Over 70 percent of performance bottlenecks are due to system infrastructure issues. By detecting and resolving these in pre-production, Load Runner saves valuable time and money.

6. Automatic Analysis: Load Runner’s Auto-Correlation wizard automatically
takes all the monitoring & diagnostics data and provides the top-five causes of performance degradation. As a result, Load Runner has the industry’s deepest and broadest analysis. Converting performance-testing results into actionable, precise data to development dramatically reduces time to resolution and allows for more testing cycles. This is a critical piece in ensuring that a high-quality application goes into production.

7. Ease of Use: Load Runner is built from the ground up for QA users. It
provides a visual scripting language, Data and autocorrelation wizards and Active Screen technology to make scripting and running of performance tests as easy as possible. Load Runner Terminology Scenario: A scenario defines the events that occur during each testing session.

Scenario defines and controls the number of users to emulate, the actions that they perform and the machines on which they run their emulations.

Load Runner

17

Vusers: In the scenario, Load Runner replaces human users with virtual users or Vusers. When you run a scenario, Vusers emulate the actions of human users working with your application. Vuser Scripts: The actions that a Vuser performs during the scenario are described in a Vuser script. When you run a scenario, each Vuser executes a Vuser script.

Transactions: To measure the performance of the server, you define transactions. A transaction represents an action or a set of actions that you are interested in measuring. Controller: LoadRunner Controller will be used to manage and maintain your scenarios. Using the Controller, you control all the Vusers in a scenario from a single workstation. Load generator: When you execute a scenario, the Load Runner Controller distributes each Vuser in the scenario to a load generator. The load generator is the machine that executes the Vuser script, enabling the Vuser to emulate the actions of a human user. Load Runner Vuser Technology

Remote agent Dispatcher

Vuser

Controller

Agent

Vuser Vuser

Load generator

Load Runner

18

Creating Scripts using Load Runner We have to create the scripts by recording the steps specified in the business scenario document. Creating the script consists of the flowing steps.

Record Vuser Script

Enhance Vuser Script

Configure Runtime Settings

Run the Vuser Script in Stand Alone Mode
Recording Vuser Scripts using VuGen We can use VuGen to develop the Vuser script by recording user performing typical business processes on a client application. Each Vuser script that you create with VuGen can directly communicate with a server by executing calls to the server API without relying on client software. In addition, when a user communicates directly with the server, system resources are not used on a user interface. This lets you run large number of Vusers on a single workstation. 1. Open the Virtual User Generator. 2. Select the Protocol for recording. 3. Click on Start recording. 4. Enter URL/Path of the application for recording.

Load Runner

19

5. Record all the steps for the specified business scenario Enhancing the scripts While recording a script or after recording, you can enhance its capabilities by adding the following functions to the script.

1. General functions 2. Protocol specific functions. 3. Standard ANSI C functions. 4. Parameterization. 5. Correlation. General Vuser functions: General Vuser functions greatly enhance the script functionality of any Vuser script. You can use general Vuser functions to measure the server performance, control server load and debugging code or retrieve run-time information about the Vusers participating in the scenario. Protocol Specific Functions: There are several library functions that you can use to enhance your script. For example LRS functions for Win Socket applications, LRD functions for database applications etc. Standard ANSI C Functions: You can enhance your Vuser scripts by adding ANSI C functions to the script. ANSI C functions allow you to add comments, Control flow statements and conditional statements so on to Vuser scripts. Parameterization:

Load Runner

20

When you record a business process VuGen generates a script that contains actual values used for recording. Suppose you want to perform script actions like query, Submit using different values from those recorded values. To do this you need to replace the recorded values with parameters. This is called as parameterization.

Correlation: Correlation is used to capture the dynamic value for a particular field at run time. The primary reason for correlating statement is 1. To simplify or optimize your code 2. For Dynamic Data 3. To accommodate unique records. You can also enhance the scripts by 1. Inserting transactions into a Vuser script. 2. Inserting Rendezvous point into a Vuser script 3. Inserting comments into the script. 4. Sending messages output. 5. Handling errors in Vuser Scripts. Configuring Run time Settings The run time settings will define the way that the script runs. These settings are stored in the file default.cfg located in the Vuser directory. Runtime settings are applied to Vusers when you run the scripts in VuGen or controller. We have to adjust the run time settings based on the no. of transactions/hour. We will be calculating this using the formula. Transactions / hour = no. of user* 60 /

Load Runner

21

Action section Execution time + Think time + Pacing Time) Run Logic

Number of Iterations: Based on the number of iterations set, Load Runner repeats all the Actions for specified number of times. If you specify scenario duration in the Controller’s Scheduling settings, the duration setting overrides the Vuser iteration settings. This means that if the duration is set to one hour, the Vusers will continue to run as many iterations as possible in one hour, even if the run-time settings set to one iteration. Pacing The Pacing Run-Time settings let you control the time between iterations. The pace tells the Vuser how long to wait between iterations of your actions.

Load Runner

22

You instruct the Vusers to start each iteration using one of the following As soon as the previous iteration ends: The new iteration begins as soon as the previous iteration ends. After the previous iteration ends with a fixed or random delay of …: Starts each new iteration, with a delay of specified amount of time after the end of the previous iteration. Specify either an exact number of seconds or a range of time. At fixed or random intervals, every … [to …] seconds: You can specify the time between iteration either a fixed number of seconds / range of seconds. For example, you can specify to begin a new iteration every 30 seconds, or at a random rate ranging from 30 to 45 seconds from the beginning of the previous iteration. Each scheduled iteration will only begin when the previous iteration is complete. The

Load Runner

23

actual amount of time that the Vuser waits between the end of one iteration and the start of the next one appears in the Execution Log when you run the script.

Log

Enable Logging This option enables automatic logging during replay. VuGen writes log messages that you can view in the Execution log. This option only affects automatic logging and log messages issued through lr_log_message. Messages sent manually, using lr_message, lr_output_message, and lr_error_message, are still issued. The Log run-time settings allow you to adjust the logging level depending on your development stage. Standard Log Creates a standard log of functions and messages sent during script execution to use for debugging. Disable this option for large load testing scenarios. If the logging level

Load Runner

24

is set to Standard, the logging mode is automatically set to JIT logging when adding it to a scenario.

Extended Log Creates an extended log, including warnings and other messages. Disable this option for large load testing scenarios. If logging is disabled or if the level is set to Extended, adding it to a scenario does not affect the log settings. You can specify additional information, which should be added to the extended log using the Extended log options: Parameter substitution Select this option to log all parameters assigned to the script along with their values Data returned by server Select this option to log all of the data returned by the server. Advanced trace Select this option to log all of the functions and messages sent by the Vuser during the session. This option is useful when you debug a Vuser script. Think Time

Load Runner

25

Vuser think time emulates the time that a real user waits between actions. For example, when a user receives data from a server, the user may wait several seconds to review the data before responding. This delay is known as the think time. VuGen uses lr_think_time functions to record think time values into your Vuser scripts. By default, when you run a Vuser script, the Vuser uses the think time values that were recorded into the script during the recording session. VuGen allows you to use the recorded think time, ignore it, or use a value related to the recorded time. Ignore think time: Ignore the recorded think time option replay the script by ignoring all lr_think_time functions. Replay the think time: The second set of think time options let you use the recorded think time.

Load Runner

26

As recorded: During replay, use the argument that appears in the lr_think_time function. For example, lr_think_time (10) waits ten seconds. Multiply recorded think time by: During replay, use a multiple of the recorded think time. This can increase or decrease the think time applied during playback Use random percentage of the recorded think time: Use a random percentage of the recorded think time. You set a range for the think time value by specifying a range for the think time. Limit think time to: Limit the think time’s maximum value to the specified value.

Miscellaneous Run-Time Settings Error Handling You can specify how a Vuser handles errors during script execution. By default, when a Vuser detects an error, it exits. You can use the run-time settings to instruct a Vuser to continue script execution when an error occurs. To do so, select the Continue on Error check box in the Miscellaneous run-time settings. You can also instruct VuGen to mark all transactions in which an lr_error_message function was issued, as Failed. The lr_error_message function is issued through a programmed If statement, when a certain condition is met.

Load Runner

27

Multithreading Vusers support multithread environments. The primary advantage of a multithread environment is the ability to run more Vusers per load generator. Only thread safe protocols should be run as threads.

• •

To enable multithreading, click Run Vuser as a thread. To disable multithreading and run each Vuser as a separate process, click Run Vuser as a process.

Automatic Transactions You can instruct Load Runner to handle every step or action in a Vuser script as a transaction. This is called using automatic transactions. Load Runner assigns the step or action name as the name of the transaction. By default, automatic transactions per action are enabled.

To disable automatic transactions per action, clear the Define each action as a transaction check box. (enabled by default)

Load Runner

28

To enable automatic transactions per step, check the Define each step as a transaction check box. (disabled by default)

Speed Simulation

Using the Speed Simulation settings, you can select a bandwidth that best emulates the environment under test. The following options are available:

Use maximum bandwidth: By default, bandwidth emulation is disabled and the Vusers run at the maximum bandwidth that is available over the network. Use bandwidth: Indicate a specific bandwidth level for your Vuser to emulate. You can select a speed ranging from 14.4 to 512 Kbps, emulating analog modems, ISDN, or DSL. Use custom bandwidth: Indicate a bandwidth limit for your Vuser to emulate. Specify the bandwidth in bits, where 1 Kilobit=1024 bits. Proxy

Load Runner

29

Proxy runtime settings are used to provide the Proxy server details to the scripts.

No Proxy: For all the Vusers make the connection directly to the internet, without using Proxy Server. Obtain the Proxy Server settings from the default Browser: All Vusers use the proxy settings of the browser on the machine they are running. This is the default option. User Custom Proxy: All Vusers connect to the Internet through a Proxy using the settings you specify.

Use Proxy: Enabling this check box provides the Address of the proxy to use and port. If the proxy requires authentication information provide this in the proxy authentication window specified below.

Load Runner

30

Executing Scenario Using Controller Scenario: A scenario defines the events that occur during each testing session.

Scenario defines and controls the number of users to emulate, the actions that they perform and the machines on which they run their emulations. Manual Scenario: Select this method if you want to build a manual scenario. You build a manual scenario by creating groups and specifying the script, the load generator, and the number of Vusers included in each group. Use the Percentage Mode to distribute the Vusers among the scripts: Select this option if you want to build a manual scenario by specifying a number of Vusers to be distributed among the selected Vuser scripts. Goal Oriented Scenario: Select this method to have Load Runner build a scenario for you. In a goal-oriented scenario, you define the goals you want your test to achieve, and Load Runner automatically builds a scenario for you, based on these goals. This scenario is mainly used for stress testing. Scheduling a Scenario Ramp up: Specifies the entry criteria for the users into the server. Load All users Simultaneously: Will load all the users at a time after the scenario is started. Start {No. of users} after every {Time}: will load the specified number of users in to the server in a given time.

Duration: Run until Completion: Will run the scripts once. If the number of iterations specified inside the Runtime Settings, Script will be executed for specified number of iterations for each user.

Load Runner

31

Run for {Time} after ramp up has been completed: Scripts will be run for specified duration after all the users were ramped up. Script will be repeated for ‘n’ number of times based on the duration specified. On selecting this option, Run Logic runtime setting specified in the script will be overwritten. Run Indefinitely: Scripts will be running indefinitely till we stop the scenario. This option will be used for conducting the Volume testing of an application. Ramp Down: Specifies the exit criteria for the users from the server. Stop all Vusers Simultaneously: Exits all the users from the server at a time. Stop {No. of users} after every {Time}: Stops a certain number of Vusers within a specified time frame. Setting Up Load Generator Machines After installing the load runner agent software on the Load Generator machine we have to configure the Agent for connectivity with the controller machine. Follow the below steps for configuring the Agent 1. Go to Programs -> Load Runner ->Advance Settings -> Agent Configuration. On selecting this it will display the following window

Select the Enable Firewall Agent option and click on Settings button. It will display all the setting related to the agent configuration. Set the following properties for configuring the agent. Local Machine Key: Specify the IP Address of the controller machine to establish a unique connection between the Controller the agent machine.

Load Runner

32

Configuring Load Generators For configuring the load Generator click on the Generator button in Controller design window and add the load generator machine by specifying the name of the machine. Click on Connect button to check whether the connectivity between the controller machine and Load generator machine can be established. If the connectivity is failed please check the following. 1. Login into the Load Generator machine ensure that Agent process is launched and running.

2. Check the network connectivity between controller machines by running the
command Ping machine_name at command prompt of the controller/ agent machine. 3. Verify the Load Generator Machine name / IP address in the load generator window. 4. Verify whether the agent machine configured using Agent configuration. 5. Verify whether any firewall exists. Setting up the Monitors Monitors Load Runner uses a suite of integrated performance monitors to quickly isolate system bottlenecks with minimal impact on the system. The suite consists of monitors for the network, application servers, Web servers and database servers. These monitors are designed to accurately measure the performance of every single tier, server and component of the system during the load test. By correlating this performance data with end-user loads and response times, IT groups can easily determine the source of bottlenecks. In addition, they can collect and manage all system performance data from the Load Runner Controller. We can configure the monitors in controller to capture these details.

Configuring Load Runner Monitors.

1. Go to Run tab of controller window. 2. Select the corresponding monitor (Web / Application / Database server) from
Available Graphs section

Load Runner

33

3. Drag and drop the monitor into any of the graphs. 4. Right click on the graph and select Add Measurements Now it will display the following window.

5. Click on Add(top) and enter the machine name of the Web/App/DB server 6. Now click on the Add (Down) button and select the required Performance counters to be measured during the scenario execution. 7. Click on OK button. If controller is not able to capture the details after configuring please ensure the following. 1. Verify the connectivity between the Load Runner Controller machine and Server Machine. 2. Verify whether you are Admin user or you have sufficient privileges to capture the performance counters from the server machine.

Load Runner

34

Analyzing Results Once the results are ready, analysis is the most important job in the whole performance test if the requirements of performance test are not defined. The main criteria for analysis should be to find out the bottlenecks and we should be able to track the transactions / users / hits that create the highest level of performance reduction. Based on the test run, Load Runner will generate various graphs and reports. This phase contains: Analysis Once the load test is completed user can correlate the various graphs to analyze the results and to identify the business scenarios causing high CPU utilization and network traffic. Monitoring It includes observation of what is happening in each tier of the application, web server, Application server and Database server. resources utilized and network delays. Tuning It includes observation and understanding of the particular query or activity which are not meeting the response time and tuning it by creating indexes, or reordering the queries and by modifying the hardware( if required). Diagnostics Performance testers have a single, unified view of how individual tiers, components and SQL statements impact the overall performance of a business process under load conditions. With diagnostics, performance testers using Load Runner can see all components touched by an end – user transaction. More over, users can also see how long each component takes and how many times it is called. With this information corresponding team can focus on resources to improve the end-user experience by targeting the most significant area of the web application and database server bottlenecks. We can analyze the system

Load Runner

35

Summary Report The summary Report provides the information about the scenario execution. It provides the information like Running Vusers, Throughput, Hits Per second, HTTP Responses per Second, Transaction Summary and Average Transaction Response time. 90 percent column indicates the maximum response time for ninety percent of the Transactions.

Load Runner

36

Running Vusers Graph

Running Vusers graph displays the number of users that executed the scripts and their status during the execution. This graph is useful to determine the load on the server at any given moment. Hits per Second The Hits per Second graph shows the number of HTTP requests made by Vusers to the Web servers during the scenario execution. This graph helps you to evaluate the amount of load Vusers generate in terms of number of hits.

Load Runner

37

Load Runner

38

Transaction Summary

Transaction summary graph summarizes the number of transactions in the scenario that failed, Passed, stopped and ended with errors.

Load Runner

39

Average Transaction Response Time

This graph displays the average time taken to perform transactions during each second of the scenario run.

Load Runner

40

Tuning Information Captured for Tuning For analyzing the results and for identifying the bottlenecks capture the following information during the Scenario execution. 1. Run the SQL Server Trace file during the Scenario Run. 2. Capture the Performance counters during the run into the Load Runner controller (if you have access to LR Monitors) otherwise use the windows performance monitor to capture the same. 3. Run the Diagnosis tools, if you find any Hardware issues.

Performance Counters to be captured 1. CPU Utilization 2. Avg. Disk Queue Length 3. Network Interface 4. System 5. Server 6. Process 7. Physical Disk 8. Logical Disk 9. Memory

Few Tips on Resources utilization of the Servers Parameter Utilization Avg. Disk Queue Length Load Average CPU Intensive Processes Swapping Paging Data Buffer Hit Ratio High Disk Activity Tips None of the CPUs should have utilization of more than 80 percent. Should not be more than 2. More than three processes should not wait for the CPU. Any non-DB, system processes utilizing CPU intensively warrant attention. Minimal to no swapped-out processes Minimize page faults More than 70 percent Disk should have less than 50-70 I/Os per second. Also, the disks should be less than 60 percent busy.

Load Runner

41

Minimal to no disk request queue. A queue of 2-4 Long Disk Queues Un-indexed Tables warrants attention. All the tables should have at least one Primary Key.

Measurement Calculations: The less time per transaction, the greater a system's throughput, as expressed by the formula derived from Little's Law: Server Thruput = # of real simultaneous users / ( Average Response Time + Think Time ) So when no think time is used, the number of virtual users can be calculated by this formula suggested by Menasce & Almeida: # of real simultaneous users * [ Response Time / ( Response Time + Think Time ) ] For example, to simulate 1,000 real users using 2 second response time and 20 second think times: 1,000 x [ 2 / ( 2 + 20 ) ] = 91

System Load is measured by a rate -- the total number of tasks performed during a certain period (an hour, day, week, month, quarter, year, etc.). Example: 300 transactions per second. There can be several levels of load: o Peak number per hour or per second is typically used for system sizing. o o Normal load levels are used to plan on-going support requirements. Low load levels are used to plan the percentage of resources which can be taken off-line for repairs and upgrades.

Load Runner

42

Measures of Variation The average measures the central tendency in a group of numbers. For example, the average of 10, 20, and 30 is 20. This is calculated by adding all three numbers together (10 + 20 + 30 = 60) and dividing this sum by the number of items added together (in this example, three).

Standard Deviation Standard deviation is a statistical measure of the extent of variation in a set of data. The standard deviation tells you the extent of the spread of individual items around the average. The number in the 90% column is shown after a run is completed in the Analysis program, not in the Controller) because it is calculated by sorting all items by value obtained during the run, then identifying the value associated the item at the top 10% of all items.

References

1. http://www.aptest.com/resources.html 2. http://www.sql-server-performance.com 3. http://www.mercuryinteractive.com/
4. http://www.qaforums.com

Load Runner

43

Sign up to vote on this title
UsefulNot useful