You are on page 1of 51

Load Testing Oracle Business Intelligence Enterprise Edition

(OBIEE) Using Oracle Load Testing (OLT) 12.x

Author @ Ahmed Awan
Customer Engineering & Advocacy Lab (CEAL) – https://blogs.oracle.com/pa

December 2014
Version 3.0

© 2014 Oracle Corporation

Introduction
This document is intended as a guide for creating load/performance test
scripts against Oracle Business Intelligence Enterprise Edition (OBIEE)
using Oracle Load Testing. This guide will assist during the script creation
process and enable the OBIEE tester to create scripts faster and more
reliably. It assumes that the person using this document has experience
working with Oracle Application Testing Suite. This document does not
necessarily cover all OBIEE transactions that have to be tested, nor does it
guarantee that the parameters mentioned in this document will perfectly
match your particular environment.

Table of Content:
- 1: Oracle Application Testing Suite (OATS) Overview.

- 2: OBIEE Load / Performance Testing Process Using OATS.

© 2014 Oracle Corporation

1: Overview
About Oracle Application Testing Suite (OATS 12.x)

Functional Testing for Web Apps

Test Manager for Web Applications

Load Testing for Web Apps

For more information on this OATS, please visit
> http://www.oracle.com/technetwork/oem/app-test/etest-101273.html

© 2014 Oracle Corporation

1: Overview
About OATS - Oracle Load Testing (OLT) Architecture

Oracle Load Testing Agents
(OATS Agent Service)

Agent 1 Virtual Users
Oracle Load Testing Server
(OATS Application Service)

Virtual Users
Agent 2
OLT
OLT Controller
ServerStats
Module
Module
Virtual Users
Agent 3

WebLogic Server

Virtual Users
Agent n

OLT Database

Oracle DB

© 2014 Oracle Corporation

1: Overview
About OATS - Oracle Load Testing (OLT) Components

• OLT Server: Main interface for configuring load test scenarios and
server monitors, running load tests and viewing results
- “OATS Application Service” is used to run the OLT Server
- Typically one Server/Controller component per deployment
- WebLogic 11gR1
- Web-based user interface accessible through IE or Firefox

• OLT Agents: Agent is used to run Virtual Users which execute load
test scripts to generate load against application under test
- “OATS Agent Service” in is used to run the OLT Agent
- Controller will start Virtual Users on configured Agent systems at run time

© 2014 Oracle Corporation

html • About Oracle Load Testing (OLT) .http://www.com/technetwork/oem/downloads/index-084446.http://download.http://apex.html © 2014 Oracle Corporation .oracle.com/pls/apex/f?p=44785:2:0:FORCE_QUERY::2.oracle. 1: Overview OATS Learning Resources Links • Oracle e-Learning Library for OATS: .R IR.com/technology/products/oem/screenwatches/ats_loa d_testing/index.html • Download latest OATS from .com/technetwork/oem/app-test/etest-101273.oracle.oracle.http://www.CIR:P2_TAGS:ATS • All About Oracle Application Testing Suite (OATS) .

Load / Performance Testing OBIEE • Design / Build Test Scripts • Perform Basic OBIEE Tuning • System Requirements: Oracle Load Testing (OLT) • Perform Load Test • Load Test Results & Analysis © 2014 Oracle Corporation .

OBIEE Using Oracle Application Testing Suite (OATS) © 2014 Oracle Corporation . Performance Test Process .

<Insert Picture Here> Design / Build Test Scripts © 2014 Oracle Corporation .

• Parameterize the Load Test Script. © 2014 Oracle Corporation . • Record the Load Test Script. • Tune and Validate Load Test Script. • Set the Thinktime in the Load Test Script. • Correlate the Load Test Script. Design & Build Load Test Scripts • Define User Scenarios.

oracle.Go to the website URL (http://<hostname>.Click ‘Sign Out’ © 2014 Oracle Corporation .Click ‘Dashboards’ menu .Enter username/password & click Login button .For example. . .com:9704/analytics) .Select dashboard ‘Competitors’ . Define User Scenarios • Define the user scenarios for the load test.

Close the browser .Launch Open Script to record the script . © 2014 Oracle Corporation . .Create new script & select Web/HTTP protocol. Here are the steps to record the script: .Click the record button to start recording.Open the browser and go through the scenario once to warm up the browser and server . Record the Load Test Script • Note: you need to have Oracle Application Testing Suite (OATS) installed to be able to record load test scripts. go through the user scenario and save the script.

© 2014 Oracle Corporation .Create new script & select Web/HTTP protocol. Record the Load Test Script • How to Use OpenScript to record the script: .

Record the Load Test Script • How to Use OpenScript to record the script: .Click “Record” button* to begin recording script © 2014 Oracle Corporation .

Set the Thinktime in the Load Test Script • Setting the right Thinktime is very critical to get the correct results because it controls the load the load generator is going to put into the OBIEE system.For example from UI: © 2014 Oracle Corporation . For example in java code: .beginStep("[3] Oracle BIEE Home". . .The Thinktime (in milliseconds) is the first parameter for the function in the script. 60000).

How to define Thinktime between each steps. Set the Thinktime in the Load Test Script • In OpenScript . In Tree View. double click on each Step Group defined: © 2014 Oracle Corporation .

such as ‘_scid’. ‘ViewState’…etc.The following rule defines the variable © 2014 Oracle Corporation . • Tip: How to define Correlation rules . and the script needs to be able to pick up the changes accordingly . What needs to be correlated is application specific. The process is called ‘script correlation’. Correlate the Load Test Script • There are dynamic contents returned from the server.

".param("_scid". http. http.param("urlGenerator".None).param("path".us.solve("Scid_2".oracle. "http://pa.+?)\".vXRfc3pu3ZmpTBa0OHOG}}")."qualified")).Login_User. © 2014 Oracle Corporation .param("CatalogTreeModel")).biconsumeruser4}}").param("action". "ii"). http. "f").com:9704/analytics/saw. "". 0.Html.Source."{{Scid_2.Login_Users. http.dll". • Where the value is substituted in the script .param("followLinks". true. Correlate the Load Test Script • Where the value is captured in the script . http. true. null. "var obips_scid=\"(. "/users/{{db. http. "UTF8").post(47. http. http. "UTF8".psr1004. EncodeOptions.postdata(http. "utf-8").querystring(http. http.param("icharset".

© 2014 Oracle Corporation .For automatic correlation. it is recommended to import the rules xml file. In OpenScript. Correlate the Load Test Script • Import Correlation Library for OBIEE: . run ‘Menu: Script-> Script Properties’ and Edit… button.

Click on Import… button and use below embedded XML (OBIEE_Correlation_Library. Correlate the Load Test Script • Import Correlation Library for OBIEE .xml attached in this pdf) © 2014 Oracle Corporation .

Apply correlation rules in the recorded script: In OpenScript. run ‘Menu: Script-> Correlate Script’ to apply the correlation rules © 2014 Oracle Corporation . Correlate the Load Test Script • After you import the correlation xml file you should see below known correlation rules for OBIEE. You might need to define your own correlation rules when needed (as per slide 19-20).

Basically. Parameterize the Load Test Script • We would like each virtual user to use a different user id. and then use it in the script where the user id is needed.Login_Users. .How to use Databanks (refer to OpenScript Help for details): © 2014 Oracle Corporation . this is done by define a parameter in the databanks (called db.Login_User in this example).

param("icharset".Login_Users.dll". the parameters appear as below: http.post(19.ceal1004. "utf-8")). © 2014 Oracle Corporation .postdata(http. http. Parameterize the Load Test Script In the Code View.param("startPage". "").querystring(http.param("Locale". http. true.param("bieehome")). "1"). "welcome1"). http.biconsumeruser4}}"). http.Login_User. http. http.param("NQUser".us.param("NQPassword". "UTF8").com:7001/analytics/saw."{{db.oracle."http://pa. null. "UTF8".

• In OpenScript. playback the final parameterized / correlated script Note: Playback script with Iterations. © 2014 Oracle Corporation . Tune and Validate Load Test Script. Select “Iterate” from the Script menu and select the Run no more than [ ] iterations option and set the iteration count to 10.

<Insert Picture Here> Perform Basic OBIEE Tuning © 2014 Oracle Corporation .

Perform Basic OBIEE Tuning • Tuning and Optimization. © 2014 Oracle Corporation . • Specific Configurations for Load Test.

OBIEE Tuning Guide @ 1333049.com/epmos/faces/DocumentDisplay?id=1333049.1 (OBIEE 11g Infrastructure Performance Tuning Guide) can be found at: https://support. it is recommended to implement best practices tuning settings stated in below document: . Tuning and Optimization • Prior to load tests execution.1 © 2014 Oracle Corporation .oracle.

OldCursorWaitSeconds is for subsequent callbacks for the query to finish. • Avoid Excessive Logging .Avoid excessive writing to log files in following BI / WebLogic Server components because this can quickly create a disk bottleneck under high users load test. © 2014 Oracle Corporation . so the load test scripts won’t error out because of not seeing the expected results: <Cursors> <NewCursorWaitSeconds>1000</NewCursorWaitSeconds> <OldCursorWaitSeconds>1000</OldCursorWaitSeconds> </Cursors> Important Note: Revert the above setting to default once the load test is completed.There are two parameters that control how long OBI PS should wait before replying ‘refreshing page’ to the browser. Specific Configurations for Load Test • OBIPS Cursor Wait Time . NewCursorWaitSeconds is the time to wait on the initial callback from the UI for a query to finish. Note: It is helpful to increase the ‘NewCursorWaitSeconds’ for performance load testing.

See below: © 2014 Oracle Corporation . set default log level for all BI System components to “Error”. Specific Configurations for Load Test • Set BIEE log level to ‘Error’ .In EM.

See below: • Lower the OBIS Query Loglevel .Make sure debug log is set to disabled in WebLogic Server for each bi_server_n managed server. Specific Configurations for Load Test • Set WebLogic Server debug log to disabled .It has reasonable low overhead in low to moderately busy system and can be kept at level 2 in production © 2014 Oracle Corporation . Loglevel 2 provided the logical query. physical query and the response times .

<Insert Picture Here> System Requirements: Oracle Load Testing © 2014 Oracle Corporation .

you should consider deploying Oracle Load Testing (OLT) on a faster server class machine with additional RAM recommended.System: x86 64-bit processor.6 GHz or faster (4 Cores) . System Requirements: Oracle Load Testing (OLT) • Number of Load Testing Server machine will vary depending on the size and configuration of your load test. Note: It is not recommended to run OLT Agent and OLT Server on same system. So for example if you are planning to run 5000 Virtual Users (VUs).Memory: 8 GB. of greater than 1000 Virtual Users.Can typically run up-to 1500 Virtual Users (VUs) per OLT Agent (64bit) system machine with following hardware specifications: .Windows 2003. 4 GB Memory) to host OLT Server (used to direct entire load test & results) . If you are running larger load tests.0 GHz. 2. © 2014 Oracle Corporation . you would need three machines running OLT Agent 64-bit version with above hardware specifications and one machine (with 2. . Windows 2008 . Windows 7.

<Insert Picture Here> Perform Load Test © 2014 Oracle Corporation .

Perform Load Test • Launch Oracle Load Testing Controller. . Select the script(s) from the appropriate Workspace © 2014 Oracle Corporation .First step is to create a load test scenario by selecting the script or scripts you want to run in your load test.

.Click the button to go into the configure all parameters page.150% © 2014 Oracle Corporation . Perform Load Test • Configure all the parameters: .Set the Thinktime range 50% .

Set Virtual user (VU) ramp up: Ramp up 2 users every 15 seconds .Click the Run Test button to start the run © 2014 Oracle Corporation . Perform Load Test • Run the test with Oracle Load Testing .

Perform Load Test View Real Time Results • OLT’s “Watch VU Grid” and “View Run Graphs” tabs allows you to see real-time results as the load test session is running: © 2014 Oracle Corporation .

Oracle Load Testing also allows you to monitor key performance diagnostics for OBIEE infrastructure using ServerStats. Perform Load Test System Monitoring Here are various monitoring options.) in real time.com/technetwork/oem/pdf/ 511863. 4.pdf). I/O. Memory. For diagnostics and monitoring OBIEE you can use OEM 12c BI Management Pack. © 2014 Oracle Corporation .. 2. When running a load test. you can use to monitor the system usage in real-time during load test run: 1. Using sar command you can monitor Tuning and Optimization performance of various in Linux / Solaris subsystems (CPU.oracle. OATS 3. (http://www. Live monitoring using dtrace in Oracle Linux / Solaris or Perfmon in Windows.

<Insert Picture Here> Load Test Results & Analysis © 2014 Oracle Corporation .

Users.Select previous test result from “Sessions” pulldown .Session Performance Report.Performance vs. . Post Run Reports • “Create Reports” tab allows users to access results from previously run load test Sessions . • Following are the main types of results you can create in Oracle Load Testing (OTL) to include in your final load test report: . © 2014 Oracle Corporation .Access the default graphs and reports. Users.Statistics (Throughput) vs. .

data rec.Amount of load being generated (# of VU’s. Performance of individual scripts (script response times.. page response times) © 2014 Oracle Corporation . page requests.). hits. etc. Session Performance Report • In “Report” sub-tab select / generate session report that includes following useful information: .

Session Performance Report • Performance of individual scripts (script response times. page response times) © 2014 Oracle Corporation .

Bottlenecks will cause script completion times to increase as you add more VU’s. Performance vs. Users • Shows script completion times (y-axis) plotted against the # of running Virtual Users (x-axis) .Invalid server responses may cause scripts to fail/terminate early and a decrease in script completion times © 2014 Oracle Corporation . otherwise graph should remain flat .

Pages/Second. Users • Shows amount of load being generated (y-axis) plotted against the # of running Virtual Users (x-axis) . and Hits/Second . Statistics (Throughput) vs. Transactions/Second.Expect to see load generated increasing as you add more Virtual Users © 2014 Oracle Corporation .“Load generated” is represented in either Kilobytes/Second (received).

there may be a queuing issue.Heavy KB/Sec fluctuations may indicate unstable servers given the current load. © 2014 Oracle Corporation . a bottleneck may exist. KB/Sec should increase. . . .As Virtual Users continue to increase.If KB/Sec remains DOES NOT increase with increasing VUs. Throughput Notes • Kilobytes/Second is the easiest throughput graph to look at: .If KB/Sec decreases given increasing users.

<Insert Picture Here> Tips & Tricks © 2014 Oracle Corporation .

x release. Tip 1: Creating Load Test Script in OpenScript for SmartView. 1. Create new script & select the “Load Testing” > “Web/HTTP” © 2014 Oracle Corporation . you need to keep the browser opened. • Some points to note: • During SmartView recording. • In OATS 12. SmartView compression is now supported.

Note the SV correlation rule is already defined in OBIEE correlation library xml file (imported in OpenScript as stated previous slide) 3. Tip 1: Creating Load Test Script in OpenScript for SmartView. Keep the browser opened and step through the transaction in the Excel to capture. 2. then type in the OBIEE SmartView URL in the format http://[machine_name:port]/analytics/jbips . Test case used for recording: select Oracle BI EE. enter login / password > browse the BI Catalog and select a SmartView report > Disconnect All © 2014 Oracle Corporation . Click “Record” button* to begin recording script.

3. Test case used for recording: select Oracle BI EE. Keep the browser opened and step through the transaction in the Excel to capture. then type in the OBIEE SmartView URL in the format http://[machine_name:port]/ana lytics/jbips. enter login / password > browse the BI Catalog and select a SmartView report > Disconnect All © 2014 Oracle Corporation . Tip 1: Creating Load Test Script in OpenScript for SmartView.

verify it worked © 2014 Oracle Corporation . set user variable and substitute the value in <usr></usr> in Java code view.g. Create CSV file. 4. Select “Iterate” from the Script menu and Select the Run no 6. Once recording is completed. Tip 1: Creating Load Test Script in OpenScript for SmartView. make sure to add Step Group for each step e. We would like each virtual user to use a different user id using DataBank. Refresh etc. Run the script to execute the more than [ ] iterations option transaction & analyze results to and set the iteration count to 10. Playback script with Iterations. Connect. 5.

© 2014 Oracle Corporation .