You are on page 1of 27

4/16/2007

Performance Analysis “True genius resides in a


& LoadRunner person who can evaluate
uncertain, hazardous &
conflicting information”

James Christopher

Rapid Review of LR

Session’s Agenda Identify important metrics to help


identify bottlenecks and…

Review: Performance Analysis


Features in LR

Out of all hundreds of


monitors, how do you
choose?
Should you monitor everything?

weak points and assisting


in capacity planning.
How do you know what
to monitor?

How long should you monitor?

1
4/16/2007

Already reviewed:

*VuGen Script Generator

Quick Review *Controller

*Performance Analyzer

VuGen create scripts of a business


process (series of transactions)

Correlate content (validation)

customize your script…


Parameterzing inputs…

2
4/16/2007

define transactions. Controller

communicates with supervises the load,


load agents…

builds scenarios. You design scenarios


based on…

3
4/16/2007

3 scenario criteria: 1. Amount of users

2. Time attributes
3. Script and Logic
i.e. Think time, duration, scaling…

important feature …view sample metrics in real

4
4/16/2007

…and capture and relay metrics back

Setup your controller


capture metric data

Analyzer Data aggregation

2. Summary form
1. Raw data form

5
4/16/2007

3. Graphical form

…view results.

Analysis Module Layout Graph


Graph
Graph Information
Viewer
PanelPane
and Data Tabs

Settings in Analyzer : Define Time


Range

6
4/16/2007

Setting the time interval


adjust
correlation
settings

Customizing the correlated metrics


Adjusting the influence factors

show or hide metrics

Other features include…

7
4/16/2007

Run Auto-Correlation Process

Merging other graphs

Filter Correlation Matches


Set granularity: Time scale

for short test durations …for long test durations


use small intervals increase the time intervals

8
4/16/2007

Import external data to existing graphs


Filtering: Set Dimension Information

Windows perf counter log

What does good analysis entail?

Functioning and Keen observation: Patterns & Trends…


complete system

9
4/16/2007

variances
Looking for causality

allotted resources

Identify relationships …
sub-systems frequency & quantity
of the load

Scalability

and evaluating correlation (relationships)… Throughput

Responsiveness

10
4/16/2007

How do we do that? Laborious way:

Task Mgr

Real time examination

Windows’
Perfmon

Capture data…

*nix’s XOSView

11
4/16/2007

In LoadRunner you can…

Manually compute the stats

Visualize the raw data and…

identify significance Use automated


Alternately…
algorithms…

Review of running analysis Quality of analysis tied to


the scenarios prepared for it.

12
4/16/2007

How many users to expect?

Defined your test objectives: Peak times?

How long can it endure?

Users Perception

Designers Perception
Know the configuration of the target
system you’re going to test…

Reconcile two perceptions

…including data pathways

Have a good conceptual understanding


and
ofthe
thecontent and protocols
system you’re going toinvolved
test…
and dependencies

13
4/16/2007

Remember to select …analysis is dependent on metrics


your metrics … captured by the controller agents

Select the performance measures


you want to track

…and
Precursor:
have relevant
data
Have enough data…

Caveats Complicated

14
4/16/2007

Rule of thumb:
Do not confuse correlation Drill down on a more granular level
Start with percentage based metrics
with causation

Run multiple tests


Establish a reliable baseline
Run the same test multiple times

Calibrate the test system …


ensure consistency

limit variability early on


and identify the performance
impact of the change.

15
4/16/2007

Summary Report

Hits per Second

Throughput
Predefined metrics: Transaction Summary

Running vusers

Average Transaction

Transactions
Vuser Summary Avg( ), Total, Time Under Load,
Compiles data of selected Response time (percentile &
metrics Distribution)

Throughput Hits/sec
Amount (in bytes) for a specific Hits on the server generated by
activity (transaction) is serviced Vusers.

16
4/16/2007

Web based metrics: predefined metrics good


for web perf analysis

Need to look deeper


Dependencies Realistic analysis

Realistic system There are key metric domains to include

17
4/16/2007

System metrics Application , User


Process metrics

Challenges of selecting the


Database Metrics* Complexity
appropriate metric

Finding the right counter


Number of
performance objects
available:

18
4/16/2007

system performance objects: Processor

Active Memory:
Processes

Disk Usage:

Logical Disk
Network:

Physical Disk

19
4/16/2007

Database Metrics: System metrics to track:

A range of 100-1000 is normal with


when process time is sustained at 80% or occasional spikes of 2000…
more, it’s a bottleneck.
Interrupts / sec
% process time
If too high, H/W device may be malfunctioning

Should be < 2

Memory metrics:
Processor Queue Length
Check the amount of threads that are running

20
4/16/2007

Should be < physical memory


Pages moving between physical memory
to paging file
Committed Bytes
Pages / sec
Should be < 20
If < 10% then too much paging

Number of pages not in RAM and instead


retrieved from the HD
Disk metrics:
Page Faults / sec

Amount of time servicing disk requests Time spent reading and writing to disk
respectively

If over 90% the disk or controller % Disk Read/Write Time


% Disk TIme is the bottleneck

Measure this metric w/ processor time to see if


processor is busy with disk requests

21
4/16/2007

Pending write and read requests to disk

Current & Avg. Disk Queue Length Network metrics:


Queue lengths in general should be low, if high
consider faster drives and higher channels

Read Bytes Network / sec


Rate at which bytes are being read from =Network Efficiency Ratio
the network

Ratio of 1:1 is processing network packets


÷ as fast as it gets them

Rec’d Bytes / sec


Bytes rec’d over the network + framing Ratio < 0.8 network packets as fast as it
set; subset of total bytes rec’d are coming faster than system’s ability to
process those packets

Use metrics provided by LoadRunner

Web Application & Services metrics

22
4/16/2007

Request Execution Request / sec


GET request / sec POST request /sec Time

Web Services ASP.NET

Total bytes: sent/ Current Users Requests Queued Total Sessions


rec / transferred

Bulk Encryption

CPU Intensive Use SSL / sec metric

SSL

Handshake

Depends on the DB you are


Selecting Database Metrics testing against

23
4/16/2007

Page Faults/sec
% total processor time

Is the CPU the bottleneck? Is memory the bottleneck?


Pages/sec
processor queue length
Cache Faults/sec

% Disk Time Page Faults/sec

IsAvg
theDisk
disk the bottleneck?
Queue Length Is memory the bottleneck?
Pages/sec

Logical Disk Counters * Cache Faults/sec

Bytes transmitted (Sent)


1/3 of website performance problems
Is networkthe bottleneck?
can be traced to database related
bytes received/sec & Totals
inefficiencies and malfunctions

Output queue length

24
4/16/2007

I/O rate for database instance

Database Metrics
More advanced analysis is required General DB metrics:
vast array of metrics to look at… DB buffer allocations

Locks & latches

I/O Related :
DB Memory
Reads
Memory Manager

Writes Transactions

Total Memory Target Memory


‘CRUD’ actions

OK,
Total
there
Memory
is enough
< Target
memory TotalNot
Memory
enough!
> Target

25
4/16/2007

Locks Latches

Lock Requests /sec


Average Latch Wait Time

Lock Wait Time


Latch Waits/sec
Average Wait Time

CPU Time (all instances)


Locks and Latches
Database Contribution to Processing Usage:
Watch out for high values
% Processor Time

Physical I/O
Cache Hit Ratio
Database Contribution to Memory Usage: I/O
Database log writes/sec
Contribution to Disk Usage:
Read Ahead Counters
I/O batch writes/sec
Working Set

26
4/16/2007

Ensure to add metrics in controller

Look beyond web metrics

Sometimes its better to over


Summary estimate than under estimate

Capture a lot of relevant data

Start out with general metrics (small) then increase the


granularity (large)

Add database monitors


Sample Work

Start with the key metrics as a baseline

27