Benchmark Report

Performance and Scalability of

iStore 11i.O + Rollup .1
(Applications 11.5.8)

October 2003

The following is intended for information purposes. The development, release, and timing of any features or
functionality described for Oracle's products remains at the sole discretion of Oracle.

Oracle is a registered trademark of Oracle Corporation. Various product and service names referenced herein may be
trademarks of Oracle Corporation. All other product and service names mentioned may be trademarks of their
respective owners.

Copyright © 2004 Oracle Corporation

All rights reserved.
Copyright 2004 Oracle Corporation Page ii 11i iStore Benchmark
 Contents

1.0 INTRODUCTION...............................................................1
Objective and Results .......................................................... 1
Audience .............................................................................. 1
2.0 BENCHMARK ENVIRONMENT.........................................2
Platform Configuration ....................................................... 2
Benchmark Workload Description ...................................... 4
Benchmark Execution.......................................................... 6
3.0 BENCHMARK RESULTS...................................................7
Response Time..................................................................... 7
Throughput .......................................................................... 9
CPU Utilization................................................................. 10
Database Connections....................................................... 12
Hits .................................................................................... 12
Transactions ...................................................................... 13
Garbage Collector............................................................. 14
TRANSACTIONS 4.0 SIZING AND CONCLUSION ..................16
STATSPACK REPORT .........................................................18

Copyright 2004 Oracle Corporation Page iii 11i iStore Benchmark

1.0 Introduction
In today’s increasingly electronic marketplace, it is more critical than ever to
be able to process online retail transactions quickly, reliably and efficiently.
Oracle Ebusiness Suite, Release 11i of iStore is a Web based store designed to
handle B2B and B2C transactions with competitive performance and
scalability. The iStore benchmark will focus on the performance aspects of the
product in terms of scalability, response times and concurrent user support.

The benchmark simulates virtual users navigating through the different

sections of the store and placing online orders to completion. It gives a clear
idea of how many users the system can support and how to scale it to handle
customer’s expected load.

Objective and Results

The goal of this benchmark is to demonstrate the throughput and scalability

characteristics of the Oracle Applications iStore product. iStore is a full
featured web based store, highly customizable and scalable.

The benchmark results demonstrated the high performance and scalability of

the iStore system. The numbers show an impressive number of transactions
handled by relatively small 4x4 and 2x2 machines for the backend and middle
tier respectively. They also demonstrate the scalable nature of the product
shown by the increase in the number of transactions with increased user load
while response times are maintained below the 5-second mark.

Audience

This paper is intended for application developers, product managers, system

architects and system administrators involved in configuring Oracle
Applications. This document is also useful for field engineers and consulting
organizations engaging in performance tuning or capacity planning.

The information provided in this paper can be used as a sizing and

configuration aide for the implementation and configuration of the Oracle
Applications Release 11i.

Copyright 2004 Oracle Corporation Page 1 of 21 11i iStore Benchmark

2.0 Benchmark Environment
This section describes the configuration of the benchmark including the
benchmark workload and the platform and environment details.

Platform Configuration

Machine Configuration
The following details the hardware configuration for the benchmark
execution:

Data Server Hardware:

Number of CPUs 4
Server Class Compaq
Host Name ap618wgs
CPU Details Intel Pentium 700MHz
Main Memory 8 GB
Operating System Red Hat Linux version 2.4.9-e.12enterprise

Middle Tier Server Hardware:

Number of CPUs 2
Server Class Dell
Host Name ap676wgs
CPU Details Intel Pentium 1266 MHz
Main Memory 4 GB
Operating System Red Hat Linux version 2.4.9-e.12enterprise

WebCache Server Hardware:

Number of CPUs 4
Server Class Unisys
Host Name ap171wgs
CPU Details Intel Pentium 500 MHz
Main Memory 4 GB
Operating System Windows NT 4.0

Load Generator Server Hardware:

Number of CPUs 4
Server Class Unisys

Copyright 2004 Oracle Corporation Page 2 of 21 11i iStore Benchmark

 Host Name ap172wgs
CPU Details Intel Pentium 500 MHz
Main Memory 4 GB
Operating System Windows NT 4.0

Benchmark Software
The following software was used in this benchmark:

Oracle Applications 11i Release 11.5.8

iStore Family Pack O + Rollup1
Oracle RDBMS 9.2.0.3
WebCache 9.0.3
LoadRunner 7.5

Copyright 2004 Oracle Corporation Page 3 of 21 11i iStore Benchmark

Benchmark Workload Description

The benchmark workload was designed to mimic online users for the iStore
product. There are two main transactions in the workload scenario, namely,
catalog and order transactions.

The catalog scenario simulates a user that browses the catalog without signing
into the store. The user performs searches and view only transactions in this
case. The catalog scenario is made up of several steps described below:

Step Action
1 choose_store()
2 browse_section()
3 quick_search()
4 item_detail()

Catalog Scenario steps

The order scenario simulates a user who intends to place an order for a
specific item. In this scenario, the user signs in and places an order to
completion through the online store. This includes the full cycle of placement
including availability checks, pricing, booking, etc. The order scenario is also
made up of several steps described below.

Step Action
1 choose_store()
2 Signin()
3 add_to_cart()
4 update_qty()
5 Checkout()
6 Signin for checkout()
7 shipping()
8 Billing()
9 place_order()
10 Signout()

Order Scenario Steps

The first cycle of the benchmark simulates B2C use case. In this scenario more
users will be browsing the catalog than those placing orders. The second cycle
of the benchmark simulates a B2B scenario where more users will be placing
orders than those browsing the catalog. The percentages are shown below.

Business Flow B2B % B2C %

Browse catalog 20 80
Shopping Cart 80 20

Scenario Breakdowns

Copyright 2004 Oracle Corporation Page 4 of 21 11i iStore Benchmark

 Since the benchmark is simulating Internet/Intranet user access to the iStore
product, all of the simulated activities were performed online. There were no
batch jobs or background processes running in this benchmark.

Data Generation
The base setup for iStore used was a vision 11.5.8 database. This is the out-of-
the-box preconfigured database supplied by Oracle Applications Rapid Install
for demo purposes.

Different accounts were used to place orders. These were created using iStore
account registration functionality.

Copyright 2004 Oracle Corporation Page 5 of 21 11i iStore Benchmark

Benchmark Execution

The benchmark was executed using Mercury Interactive LoadRunner

software, by simulating web users submitting http requests. Different number
of users was simulated in each iteration of the test to illustrate the scalable
nature of the iStore product.

In every iteration of the B2C simulated scenario, some users were browsing
catalog while others were placing orders according to percentages shown in
the workload section.

Copyright 2004 Oracle Corporation Page 6 of 21 11i iStore Benchmark

3.0 Benchmark Results
This section summarizes the execution statistics of the benchmark. The
following topics are addressed:

• User Scalability and Performance

• CPU Scalability
• Throughput
The following metrics were recorded for all iterations of the test:
Metric Description
Response Time Average, 90th %ile and maximum
Throughput Number of transactions per second
CPU Utilization Measured over all different tiers
Memory Consumption Measured over all different tiers
Database Connections Number of database connections used for the
different user counts
Roundtrips Traffic between middle tier and database
SQL Stats Multiple statistics about database activity

All the Statistics are taken for Steady state (excluding the Ramp-up and Ramp-
down time) of 25 Minute Run of the Scenario. Fixed think time of 10 seconds
was inserted before every scenario step.

Response Time

The graph below shows the average as well as 90th percentile response times
for the users. The response time is for the whole scenario. The graph shows
slight degradation to the user response time as more users come into the
system. The response time recorded is for the whole scenario. During the
multi user runs average response time per page did not exceed 5 seconds. All
graphs includes fixed think time of 10 seconds between steps.

Copyright 2004 Oracle Corporation Page 7 of 21 11i iStore Benchmark

 Average Response Time for Catalog

45

Response Time 43

41 41.339
40.387 40.419 40.585

39

37

35
20 40 80 120
No. of users

Average Response Time for Order

140

130
Response Time

120
117.013

110 108.93
106.864 107.164

100

90
5 10 20 30
No. of users

Copyright 2004 Oracle Corporation Page 8 of 21 11i iStore Benchmark

 Response Time (90%) for Catalog

45

Response Time 43

41 40.828 41.039
40.426 40.516
39

37

35
20 40 80 120
No. of users

Response Time (90%) for Order

140

130
Response Time

125.036
120
112.082
110 108.439 108.802

100

90
5 10 20 30
No. of users

Throughput

The following graph shows the total number of bytes/second served from the
web server.

Copyright 2004 Oracle Corporation Page 9 of 21 11i iStore Benchmark

 Throughput

250,000

Throughput ( bytes/second ) 200,000 200,043

150,000 141,718

100,000
71,250
50,000
35,852
0
25 50 100 150
No. of Users

The above graph shows almost linear scalability of the web server measured
by bytes/sec it is serving to the clients. The Throughput doubles everytime we
double the user count.

CPU Utilization

The following graphs shows the CPU utilization for each run. The first graph
shows CPU utilization on the middle tier. At the 150 user load, we were
running 2 load balanced JVMs for Jserv.

Mid-Tier CPU Utilization

35
30.25
CPU Utilization Percentage

30
25 25.04
20
15 13.83
10
8.4
5
0
25 50 100 150
No. of Users

Copyright 2004 Oracle Corporation Page 10 of 21 11i iStore Benchmark

 The second graph shows the CPU utilization on the database machine. The
graph also shows that the iStore scenarios are database machine bound since
the average DB machine utilization is on average 56.7% higher than that of the
middle tier.

DB CPU Utilization

70
66.6
60
CPU Utilization Percentage

50
43.02
40

30
22.78
20
12.4
10

0
25 50 100 150
No. of users

The CPU Utilization on the machine which is running Webcache is shown below. The CPU
Utilization increases with the increase in no. of users.

Webcache CPU Utilization

30
CPU Utilisation Percentage

25 24.08
20
16.73
15
12.43
10
7.66
5

0
25 50 100 150
No. of Users

Copyright 2004 Oracle Corporation Page 11 of 21 11i iStore Benchmark

Database Connections

This graph shows the No. of JDBC Connections used by the scenario. These
are the connections to the database while running the scenarios for different
sets of users. This graph proves that every user is not using a dedicated
connection on the database but they share the connections. This especially true
in the B2C scenario since most of the users are doing catalog browser rather
than order transactions.

DB Connections

30

25 25
DB Connections

20 20

15 14

10
8
5

0
25 50 100 150
No. of Users

Hits

The below graph shows the no. of Hits per second. This is similar to the
throughput graph. The number of hits doubles with increased user counts.

Copyright 2004 Oracle Corporation Page 12 of 21 11i iStore Benchmark

 Hits

35
32.262
30
Hits per second
25
22.331
20
15
10 11.271

5 5.641

0
25 50 100 150
No. of Users

Transactions

The graph below displays the scalable nature of the iStore product in the B2C
scenario. As user count increases, the system is able to handle more
transactions. The rate of transaction increase is linear relevant to the increased
user count.

Transactions

7000

6000 6130
No. of Transactions

5000

4217
4000

3000

2000 2112

1000 1059

0
25 50 100 150
No. of Users

Copyright 2004 Oracle Corporation Page 13 of 21 11i iStore Benchmark

Garbage Collector

While running the flow for different number of users we recorded the Total
time spent in GC activity , No. of Full GC and Minor GC’s and Total Memory
Freed. The relationship is shown in the graphs below.

This Graph shows the Total time Spent in the GC activity for the complete run
of the flow

Time Spent in GC

12
Time Spent in GC ( sec )

11.01
10
8 7.51
6
4
3.27
2 1.97
0
25 50 100 150
No. of Users

No. of Full GC

4.5
4 4 4
3.5
No. of Full GC

3
2.5
2 2 2
1.5
1
0.5
0
25 50 100 150
No. of Users

Copyright 2004 Oracle Corporation Page 14 of 21 11i iStore Benchmark

 This graph shows the Total number of Minor GC’s happened during the flow.

No. of Minor GC

400
371
350

300

No. of Minor GC
250 243
200

150

100
83
50 45
0
25 50 100 150
No. of Users

The below graph shows the total memory freed by the GC during the flow.
For 25 and 50 users runs we were running the single JVM for Jserv and for
100 and 150 users we were running 2 JVMs for Jserv.

Total Memory Freed

40000
Memory Free by GC (MB)

35000 35607
30000
25000
23321
20000
15000
12748
10000
6911
5000
0
25 50 100 150
No. of users

Copyright 2004 Oracle Corporation Page 15 of 21 11i iStore Benchmark

Transactions 4.0 Sizing and Conclusion
The results of this benchmark demonstrate that the Oracle iStore Release 11i is
capable of sustaining large volumes in a reasonable time window on a
reasonably sized machine while maintaining a high degree of scalability.

Below are the details for sizing the middle tier and the database machines for
a Linux platform:

Application Tier

Memory Requirements

Java Virtual Machine (Jserv)

No of Java Virtual Machine 3

Average memory per Jserv Process 18MB shared and 103MB non shared

Average number of Java threads 178

Apache httpd

Average Apache Listeners per User 0.45

Average Memory Per Apache Listener 8 MB shared + 2 MB private

CPU Requirements

Average Business Event Per Hr Per 18810

CPU
191 (assuming 98 BE/user/hour)
Average Number of conc. users per
CPU

Database Tier

Memory Requirements

Database

Average Shadow Processes Per User 0.35

(Procs/conc User)

Average Memory Per Shadow Process 14 MB Private

Copyright 2004 Oracle Corporation Page 16 of 21 11i iStore Benchmark

 CPU Requirements

Average Business Event Per Hr Per 4178

CPU
42 (assuming 98 BE/user/hour)
Average Concurrent users Per CPU

Copyright 2004 Oracle Corporation Page 17 of 21 11i iStore Benchmark

StatsPack Report
Please contact iStore development if you are interested in the StatsPack Report.

Copyright 2004 Oracle Corporation Page 18 of 21 11i iStore Benchmark