Opensta Performance testing tool

Wednesday, January 2, 2008

Comparison of LoadRunner with OpenST A OpenSTA FAQ
Comparison Between LoadRunner And OpenSTA
New in HP Load Runner 9.10
1.Protocols from Opensta Performance testing
tool
The communication protocols that can be captured, manipulated and replayed by the
tool. For those Researching, Learning and
Using OpenSTA
Load Runner
from Opensta Performance testing
Many supported. Protocols are charged per item. Has a multi-protocol recording tool
feature.
HTTP throughput in OpenSTA Results
Graphs
Opensta from Opensta Performance testing
tool
HTTP 1.0 / 1.1 / HTTPS (SSL) only.
Performance testing strategies
2.Playback functions
from Opensta Performance testing
Replaying of the script and script debugging facilities. tool
Rendezvous Point in OpenSTA
Load Runner from Opensta Performance testing
tool
Extended logging supports view of parameter values and Server messages. Also view
and comparison with ‘recorded’ version of web page view and client response load testing free software testing tools
messages. Debugging facilities in script generator, step and breakpoints. from Opensta Performance testing
tool
Opensta Comparison of LoadRunner with
Similar playback facilities, but no integrated comparison function. The debugging OpenSTA
functions are in the controller, including set break points and single stepping. from Opensta Performance testing
tool
3.Scripting language openstasearch
from Opensta Performance testing
The medium used to represent the captured protocol data and manipulate the data
for play-back. tool
Handling Viewstate in Opensta
Load Runner
from Opensta Performance testing
Uses standard syntax for “C” & allows C function libraries to be added. Has extensive tool
customised functions for the different protocols supported by the tool.
Remote recording in opensta
Opensta (IE/Mozila Firefox )
from Opensta Performance testing
SCL, it uses a proprietary, “BASIC” like language that has special automation
scriptingfacilities. Is limited in available functions, such as string manipulation and tool
supports direct DOM addressing. View all »
Extensibility

The ability to increase the functionality of the tool.

1.Load Runner
Additional TSL or “C” function libraries, limited to functional capabilities of the tool.

2.OpenSTA
SCL script modules can be defined in â??Includeâ?? files. Open Source therefore new
tool functionality can be added using C++.

Scripting Interface
The interfaces supplied by the tool application for the purpose of script editing.
1.Load Runner

Captures in several modes, high level context based and low level HTTP view. Has
both a graphical tree structure & a script view. Script view has function sensitive
entry.

converted by Web2PDFConvert.com
2.OpenSTA

Has low-level HTTP protocol view and provides graphical tree representation of the
â??Document Object Modelâ?? (DOM) structure. Viewable captured HTML rendering
and addressable server-header table. Some language sensitive, syntax colour coding
functionality.

Correlation
The task of substituting values in dynamic data to enable successful playback.

1.Load Runner
Automated correlation faculties. Including during recording, after recording and
comparing recordings with playback results. Not available for all modes of capture.
2.OpenSTA

Manual correlation using graphical interactive â??DOMâ?? structure. Facility to

automatically generate script code to aid variable substitution.
Cookie Management

Detection, recording and playback of HTTP cookies. Both tools need additional code
to manage JavaScript generated cookies.
1.Load Runner
HTTP header cookies are managed automatically and can be manipulated manually if
required.
2.OpenSTA
HTTP header cookies are managed automatically and can be manipulated manually if
required.
Parameterisation
Automatically changing dynamic data values for more accurate emulation of real
users. Often essential for session management.
1.Load Runner
Extensive facilities for data entry including wizard interface to DB interrogation. No
standard function to lock data sources and maintain uniqueness of concurrently
accessed data across distributed tests.

2.OpenSTA
Extensive facilities for data entry including wizard interface to automatically generate
test data. Standard functions for sequential, random and pseudo-random data-file
access. Has standard common locking facilities for maintaining uniqueness of
parameters for an individual load injector or across all injectors on a distributed test.
Controller
Application that manages and conducts a test.
1.Load Runner
Facilities for real-time monitoring. Automatic scenario generation. Individual control of
vusers, scripts and groups of scripts. Scheduling, percentage runs & iterations.
2.OpenSTA
Facilities for real-time monitoring. Simple drag & drop multi-scenario test configuration
supporting modular scripting and addition of new scenarios/Vus at run-time. No
automatic scenario generation. Allows test-wide and user-specific HTTP monitoring
and debugging while multi-user load in progress.
Monitoring
Resource usage information is captured during execution.

1.Load Runner
It can be shown during execution and used to build performance reports. Many
supported. On-line graphics during execution. Supplied with Apache, Netscape & IIS,
other monitors are charged per item. Results used for reporting. New facility to allow
remote users to monitor real-time results via a browser based interface. Note: To
monitor through a firewall requires TCPIP access through a specific port. Future
versions of LoadRunner should use HTTP messaging & avoid this issue.
2.OpenSTA

converted by Web2PDFConvert.com
Supports Integrated real-time, graph view of Windows NT/2000 Performance
(Perfmon) & SNMP collectors. Various measurements of test progress including VU-
specific, custom status and activity information. Web-relay allows monitors to run on
remote machines beyond firewall. On-line graphics during execution, & monitored
results used for reporting.
Distributed tests
The ability to distribute the generation of load across multiple load-injector machines.
1.Load Runner
Supports multiple load-injectors managed by a single controller.

2.OpenSTA
Supports multiple load-injectors managed by a single controller. Uses TCP/IP if on the
same network or the Web-relay feature uses HTTP to control load-injectors located
within remote DMZs.

IP Spoofing

The ability to emulate the behaviour of different IP addresses accessing a system.

Particularly useful with Load balancing systems.
1.Load Runner
Supports IP spoofing including automated router updates for IP forwarding.

2.OpenSTA
No inbuilt features to enable IP spoofing.

WAN/LAN emulation
The ability to emulate the behaviour of different network infrastructures during a
test.

1.Load Runner
New feature added to version 7.6. Allows the emulation of latency, packet loss, link
faults & dynamic routing effects over the LAN used in a test. Requires a special
license.
2.OpenSTA
No inbuilt features to enable WAN/LAN emulation.
Caching
The ability to emulate the caching of pages as performed by a web browser.

1.Load Runner

Can control browser cache emulation during playback and control setting for each
individual vuser.
2.OpenSTA

No specific faculties, although can be emulated with simple script code.

User connection speed emulation

The ability to emulate the different network speeds that can be used by real users.

1.Load Runner
Can emulate different network speeds during playback

2.OpenSTA

No inbuilt features to emulate user-connection speed emulation.

Distributed/Remote load generation

To generate large loads additional load generator machines are required. These
machines must be centrally controlled.

1.Load Runner
Can control multiple load generators and collect results.
Can control load generators located at remote networks (through a firewall) if

converted by Web2PDFConvert.com
required using HTTP ports.
2.OpenSTA

Can control multiple load generators and collect results.

Can control load generators located at remote networks (through a firewall) if
required using HTTP ports.

Reporting & analysis

The facilities to examine and investigate the results of a test including timers and
monitored resources and display the results in graphical format.

1.Load Runner

Sophisticated & large range of charts and graphs with overlay facilities. -Automatic
report generation into MS-Word. Analyser is a separate application that can be
distributed to users.

2.OpenSTA
Simple charts and graphs sufficient for analysing key load related statistics and
resource usage monitors. Resource usage monitors supports graph overlays. Can be
exported to Microsoft Excel. No license restrictions on OpenSTA distribution thus
stats can be viewed by any user with access. -Free tools and excel macros available
through public user-forums.

Scalability

The ability of the tool to generate numbers of virtual users and the corresponding
resource usage. Actual resource use depends on the number, size and complexity of
the scripts.
1.Load Runner

Resource limitations are number of threads and RAM. Approx 1 vuser per ½ Mb RAM
for NT/W2K. Windows 95, 98 & Unix are less efficient. Max. approx 1,500 VUs per PC.

2.OpenSTA
Main resource used is RAM. Tested for simple ASP pages, reaching up to 3,000 users
on load generator of 1Gb RAM on a single P4 processor & W2K. Unconfirmed report of
a limit of 1664 Vus per Win2K machine for complex scripts. Suspected thread
limitation. No licensing limitations.

Initial Costs

Purchase costs of software and licenses, excluding upgrades or support.

1.Load Runner

From £16,000 for basic package with no virtual users. Additional charge for each
protocol and monitored resource and for virtual users.

2.OpenSTA
FREE¦ download from SourceForge via www.OpenSTA.org. Downloads available:
Previous versions; Automatic installer or current source code (with simple build
instructions for MS C++ Visual Studio 6).
Costs of Virtual users
Most commercial tools charge on the basis of the number of virtual user available.
Extra hardware is an additional cost.
Prices vary but for the purpose of expectation: additional vusers cost from £10K for
25 VUs to £66K for 1000 VUs. Also temporary VU days at £3.50 each per day (min
1000). This is not a quoted price.

FREE. No licensing limitations.

Support & consultancy

The support services available for the tool and some costs.

1.Load Runner
By M.I., approx 1/5 of initial cost per annum. Includes upgrades. MI & their partner
companies also provide consultancy (including radiumtesting.blogspot.com).
2.OpenSTA

Various independent resources. etest associates charged from £50 per incident for

converted by Web2PDFConvert.com
remote tech support. Consultancy rates available on request.Numerous online
resources including web and e-mail forums. Upgrades are free (approx every 3-6
months)

Training
The training services available for the tool.

1.Load Runner
MI has a range of courses from around £400 per day per person. Many partners also
provide training.
2.OpenSTA

Specialist companies provide tailored training, prices vary.

System requirements

The operating systems required to host the tool. (Not the OS of the system under
test).

1.Load Runner
MS windows 2000, NT4 (sp6a),XP-Pro (also 95 & 98 cannot run multi-threaded) -
Load generators also support limited generator functionality on: Unix: HP, Solaris,
Linux .

2.OpenSTA
MS windows 2000, NT4 (sp5+), XP-Pro.

Hardware requirements.
The hardware requirements to host the tool. (Not the hardware of the system under
test).
1.Load Runner

Min: Pentium 350 & 128M - Load generators: Pentium 1GHz & 1 MB per VU.

2.OpenSTA
Min : Pentium 200 & 80MB RAM. Pref: Pentium 500MHz+ & 128MB+ RAM.

Access to source code

Availability of the source code of the tool itself.

1.Load Runner

Unavailable.
2.OpenSTA

Open Source GNU public license - ‘C++’ Language.

A users opinion.
Opinion of Internet performance engineers who have experience with both tools.

1.Load Runner
Has a very user friendly user interface and fantastic monitoring & results analysis.
Automatic correlation and improved script recording facilities can aid productivity.
Very flexible scripting functions & great help documents. Complexity of options and
layout of controller are drawbacks. The availability (at a cost) of numerous protocols
for recording is a significant strength.
2.OpenSTA

Easy to use interface and excellent scalability. Inbuilt results analysis is slightly
limited in comparison to LoadRunner. The captured data is open and can also be
exported to Excel in two clicks. Drag & drop scenario set-up and control is very
intuitive and easy to interact with, simplifying creation of scenarios with modular
scripts. Manual correlation can be a headache but is eased slightly by using third-
party ‘Diff’ tools and the inbuilt GUI DOM addressing feature. A relative shortage of
standard script language functions but enough to get most HTTP load testing jobs
done. If not, with “Includes” support and the availability of the source code, it is very
extensible
Posted by free testing source at 12:43:00 AM

converted by Web2PDFConvert.com
0 comments:

Post a Comment
Links to this post
Create a Link
Newer Post Home Older Post

Subscribe to: Post Comments (Atom)

converted by Web2PDFConvert.com