Professional Documents
Culture Documents
Sap PT PDF
Sap PT PDF
By
SURYAKANTH KELMANI
Sr. Architect
Performance Testing & Engineering
About Author:
I am Ex Wipro and have been working in the performance testing & engineering for the past 12
years in large projects like SAP, Oracle eBusiness, Java Documentum, Siebel, IBM WCS eCommerce
and other technologies. Major clients worked with National Grid UK, Southern Water, BP, PwC,
SHELL, Morrisons, I live in London with my wife and a lovely son. I dedicate this document to my
parents and elder brother Dr. Chandrakanth Kelmani
@2013
TABLE OF CONTENT
1. INTRODUCTION......................................................................................... 3
1.1 OBJECTIVE .................................................................................................................................. 3
1.2 PERFORMANCE TESTING FRAMEWORK ............................................................................. 3
2. PLANNING .................................................................................................. 4
2.1 BUSINESS STREAMS: ................................................................................................................ 5
2.2 TEST ENVIRONMENT................................................................................................................ 8
2.3 TEST DATA MANAGEMENT .................................................................................................... 9
2.4 TEST TOOL & SETUP ............................................................................................................... 10
4. EXECUTION.............................................................................................. 15
4.1 EXECUTION OF ONLINE USERS – In SILO .......................................................................... 15
4.2 EXECUTION OF BATCH JOBS & REPORTS – In SILO ........................................................ 15
4.3 APPROACH FOR BATCH JOBS PERFORMANCE TESTING............................................... 15
4.4 EXECUTION OF ONLINE USERS AND BATCH JOBS & REPORTS (Combined) .............. 17
4.5 RUN TIME SETTINGS FOR EXECUTION: ............................................................................ 17
4.6 SCHEDULE BUILDER ............................................................................................................. 18
7. SAMPLE GRAPHS..................................................................................... 24
7.1 UNIX RESOURCE GRAPH ....................................................................................................... 24
7.2 EXECUTIVE SUMMARY GRAPH ........................................................................................... 25
7.3 AVERAGE RESPONSE TIME GRAPH FOR T CODES .......................................................... 25
7.4 BATCH JOB/INTERFACE SAMPLE REPORT ........................................................................ 26
8. APPENDIX ................................................................................................. 26
A. LOADRUNNER HARDWARE & SOFTWARE .................................................................... 26
B. SAP T CODES & ITS FUNCTIONS....................................................................................... 26
C. SAP WEB CORRELATIONS ................................................................................................. 27
D. REFERENCES ......................................................................... Error! Bookmark not defined.
1. INTRODUCTION
1.1 OBJECTIVE
The objective of this document is to detail the best practice, approach & plan
to be followed for the Performance testing of SAP application and to make sure that
the team member who is going to carry out performance test activities can quickly
refer and implement the best practice followed across all phases of performance
testing.
This document is intended to guide further Volume Test Planning, tool setup,
Script Preparation, Execution & Result analysis activities of Performance test phase.
2. PLANNING
Performance test approach document will specify the approach taken to carry
out performance test, scope of testing, environments and tools used, inputs,
dependencies and deliverables of the Performance testing phase.
During this planning phase please note that there will be other parallel
activities that can be carried out, like identification of environments as well as the
tools for performance testing, test data analysis, work load profiling etc. Performance
testing is normally carried out on an environment that is representative of the
Production environment in terms of hardware, software configuration and database
size. Performance test tools will be used to generate scripts and simulate user load on
the system under test. Please note that HP Load Runner will be the best tool that can
be used for SAP performance testing as it’s also been widely used & recommended by
SAP.
It is necessary to have Production size data in order to get accurate test results.
There is a dependency on the other team like Data Migration to provide the required
volumes of data for carrying out performance testing. During planning phase, test data
SAP system has several Business Streams like Sales & Distribution, Financial
Accounting, Supply Chain Management, SRM, Record To Report RTR, HR, Real
Estate, Material Management etc and these are called business streams in SAP
language and given below are the example for 4 business streams identified for one of
the Energy & Utilities client for performance testing.
The following 4 streams ( for example, it could be more ) were identified for
performance testing of SAP application, each stream’s transaction codes ( T codes)
were identified with critical and most frequently used on daily basis also data
intensive in nature as shown in the below tables for each streams. Again these should
be put them into Online and batch jobs category.
Note: Each benchmark user has his or her own master data, such as material, vendor,
or customer master data to avoid data-locking situations.
Dialog Steps
0. Logon 10. Choose Enter
2. Call /nVA01 (Create customer order) 12. Choose [F20] (Posts goods issue)
User interaction steps 2 - 16 are repeated n times (15 user interaction steps --> min.
150 sec. duration).
Dialog Steps
0. Logon 11. Select first line
4. Create general ledger account item 15. Select item 5 of the list
10. Enter data and choose Execute 21. Confirm log off
User interaction steps 2 - 19 are repeated n times (15 user interaction steps --> min.
180 sec. duration).
Seven different business transactions are launched per loop (steps 2 - 8). Every loop is
repeated n times for each user. The user think time is 10 seconds; thus, one loop takes
at least 70 seconds.
The Materials Management (MM) Benchmark takes you through a series of steps to
create a purchase requisition for five materials (transaction ME51N), a purchase order
for the five materials (ME21N), a goods receipt (MIGO), and an invoice (MIRO) for
the purchase order
Dialog Steps
0. Logon 10. Choose Execute
2. Call /nME51N (Create purchase 12. Call /nMIRO (Create invoice), enter
requisition) company code
9. Enter data
User interaction steps 2 - 16 are repeated n times (15 user interaction steps --> min.
150 sec. duration).
Like online critical transactions as depicted above, I have also mentioned some of the
batch jobs names which we identified as critical for the performance test execution
during one of the SAP performance testing project.
Ideally, the simulated load and the hardware used for the volume test should
be identical to the load and hardware of subsequent production operation. However,
for various technical, organizational or economic reasons, it is not always feasible to
simulate the full load expected in subsequent production operation. In addition, it is
never really possible to execute the test on hardware exactly identical to the hardware
planned for production.
But, as a best practice performance testing should be carried out on an
environment which is mimicking close to the production environment hardware and
software configuration, the next better option is to execute performance test on
production environment before it’s Go Live, i.e. this should happen at least 6 weeks
before the Go Live so that you can make use of the best. Please note that don’t try to
do performance testing on any other environment which is not at all close or
equivalent to production like.
Important Note: Make sure that the hardware used for the volume test is
exclusively reserved for this purpose while the volume test is run. In-parallel usage of
the same hardware or system(s) for other purposes (as, for example, training system, )
can have disastrous consequences for the volume test and leads to misinterpretation of
its results.
Note that the SAP system will exchange data with a number of internal and
external non-SAP systems, i.e. Interfaces and Non SAP systems like BACS, PAY
Matrix, Billing System, Cyber Source etc.. In order to carry out Performance testing,
it is essential to connect to the test environments of all the systems identified during
testing.
The data required for Performance testing should be identified during the
Performance test planning phase. The data to be set up within SAP as well as the data
required connecting and test with external systems will be defined.
Since database tables (especially transaction data tables) start to grow after
start of production, database accesses that are not fully specified tend to get slower
and more resource-expensive. If the volume test is performed with almost empty
database tables, the response times in subsequent production operation will be higher
than measured in the volume test. Therefore, for the volume test to be meaningful,
you need to populate the database tables with representative data that is both master
and transactional data.
We must understand & set the expectation to the stakeholders that which team
is going to provide all valid transactional data and who is going to create various user
ids and passwords with proper roles & authorizations for the realistic simulation of
user load. For example all HR user ids can’t have Sales & Distribution Roles, and all
SCM user ids may not have HR roles and authorizations.
Some of the example below shows what data is required during performance testing..
• Master Data,
o Customer Account Number
o Sold To Party
o WBS Cost Element
o Material Number
o Employee Number etc.
• Transactional Data
o Bank A/c number
o User Name
o GL Account Number etc.
• User Data
o Delivery Date
o Material Price
o Customer Name etc.
Please note that the user ids and password for emulation of the load should be created
by the Tech arch team or BASIS team (SAP back end administrators are called as
Basis), make sure that the team creates the user ids with appropriate naming
conventions so that each users can be identified in the back end of the system during
performance testing as shown in the examples below.
Any data (Transaction data, user data etc) that is required during performance testing
will be provided by each business streams.
The diagram below shows that the performance test lab setup which make use
of 4 Load injectors (Load generators) with one Loadrunner Controller, all the m/c are
connected to each other on a business LAN which has access to SAP system, each
system has the 2GB RAM with Intel CPU speed 2.8GHz.
Please note that the machine running a scenario with SAPGUI Vusers must
have a Loadrunner typical installation. Running Vusers from machine with only the
controller installed is NOT SUPPORTED, and each Loadrunner systems (Injectors
and Controller m/c) should have the SAPGUI client installed so that they can talk to
back end SAP ECC system.
To rectify the above mentioned issue and allow more SAP users to be run, it is
recommended that multiple Terminal Server sessions be run on the load generator
machine (Injector) and relate each terminal server session to the load generator. This
means a Terminal Server must be running on the load generator.
Please note that you can run maximum of 30 to 50 SAPGUI users on each
terminal session, MS Windows 2003 server Standard edition should be installed on
each Load Generators machine in order to run terminal sessions, by default you can
have three terminal session run on each machine, so with this 3X50=150 total users
can be run each m/c, if you want to run more sessions then you need to buy the
license from MS vendor in order to increase the session which helps you to run more
number of SAPGUI users.
Let us recap the following points in order to complete the Planning phase,
Before you start recording in the tool, you need to set certain parameters in
SAP server in order to generate the script
b) For F4 To open F4 help in model dialog boxes, the following procedure must be
completed by a SAP administrator.
Choose Help>Settings. Click the F4 Help tab.
In the Display section (Bottom Left), choose system defaults.
In the display portion of the system defaults sections (bottom right). Select the
Dialog. Click on the pencil icon first before you can change the sections. Click on the
icon again to save lock the selection as shown in the screen shot below
Once the above settings are configured on the SAP App Server, please select
SAPGUI protocol for creating SAPGUI scripts and SAPWEB protocol for creating
Enterprise Portal scripts. The scripting for SAPGUI users and SAPWEB users are
quite simple and not to worry much about scripting part as its 80% direct record &
Replay with very few code enhancements in SAPGUI, and few complex correlations
in SAP web EP applications. There is not at all correlation concept in SAPGUI except
capturing the o/p of one process and input the same value for next process, for this
you can use Loadrunner sapgui_get_Status_bar () functions.
Once the test scripts are developed and enhanced to meet the business
requirements, parameterise various data, correlating the dynamic values etc, these
scripts should be run for various iterations with different data inputs and cross check
with the records created in the back end of the system. Once you have confirmed that
all scripts are doing what its intended to do, then get these scripts run with the
business team to confirm that they are in line with the NFR and satisfies their
requirements.
At this point of time, ask the Functional team to identify the Batch & Schedule
the batch jobs for the batch execution window, usually these batch jobs are scheduled
and executed by Functional Team either manually or by using some tool like
Controller M ( is a batch job work load scheduler tool )
4. EXECUTION
The approach for performance test execution will be carried out in the
following manner, and it should look like this as shown below.
Approach
Capture batch job baseline data of selected interface from the current ECC
system.
All the critical batch jobs which are to/from the SAP system should be
considered ( including third party systems)
Trigger Job on SAP ECC system using Control M for reference measurements
Use the input file and process in ECC XX Environment
Trigger Job in ECC XX system using Control M for validation
Tuning if required
generate performance report
*Please note that Control M is batch job scheduler tool like AppWorkx
A typical 24 hrs batch jobs approach is depicted in the below diagram, which
illustrates the batch jobs executions in SAP and Non SAP integrated environment,
please note that the batch jobs should be given much importance as process large
amount of data to/from the SAP & third party applications and they need to meet the
performance requirements/KPIs as per the NFR document.
4.4 EXECUTION OF ONLINE USERS AND BATCH JOBS & REPORTS (Combined)
After completion of performance test execution for online users and Batch Jobs, we
need to combine both the scenarios Online Users & Batch jobs to emulate the real scenario
and measure the system performance of SAP R/3.
The below screen shot shows how the scenarios with the load generator look
like, please note that all the load generators sessions are running on MS terminal
server sessions and each session is mapped with their respective names like as shown
in the screen shot below.
The below screen shot shows the various RunTime Settings for the scenarios
executions, its important to apply these runtime settings to in order to execute the
scenario successfully.
Wait time: This is the time a user request sits in the dispatcher queue.
Roll-in time: This is the amount of time needed to roll user context into the
work process.
Load time: This is the time needed to load from the database and generate
objects like ABAP code, CUA, and screen information.
Processing time: This is the time spent in the work process executing ABAP code.
Database time: Starts when a database request is put through to the database
interface; ends when the database interface has delivered the
result.
CPU time: This is the total CPU time used by the SAP R/3 work process.
The SAP monitor displays statistics about the resource usage of a SAP R/3
system during the scenario or session step run. This monitor is appropriate for SAP
R/3 server and SAPGUI (version 6.20 or later) environments.
SAP GUI transactions help the tech arch team and performance team to
quickly isolate the performance issues within the SAP R/3 infrastructure, and to
authoritatively delegate problem resolution to the right SAP application teams.
Database layer shows the amount of time spent by an end-user business process or an
individual dialog step in the database tier.
System layer shows a breakdown of time the dispatcher is waiting for a free work
process, time taken to load or generate ABAP programs by the work process, time
taken to associate the user context with a work process, and time taken to save user
context data on the outbound path.
Processing layer shows the amount of actual time taken by the work process to
process the ABAP programs and SAP application server.
Below diagram shows the break down of each layer and this should help the team to
find out the root cause of the bottleneck.
UNIX monitor gives the system resource parameters of Unix server box, in
order to run UNIX monitor successfully on Loadrunner controller, you need to run
rstatd daemon service on Unix server box, you can ask database admin team to set this
service running, or else you can ask him to carry out the following steps as shown
Where inet_pid is the pid of the inetd process. This instructs the inetd to rescan the
/etc/inetd.conf file and register all daemons which are uncommented, including the
rstatd daemon.
PS: You can configure the remaining monitors like Oracle Database and Windows
resource monitor just by adding their respective IP addresses.
Transaction ST03n records the response time of the system and of individual
transactions. It can be used to see how fast the system is responding. ST03n also divides
response times into CPU time, wait time, load time, and database request time. These times
will be evaluated as a percentage of Average Response Time (ART) for each server during
each test so that it can be determined where increases occur.
important statistic on the screen. Free space measures how full the buffer is, and swaps show
how many times data had to be removed from the buffer to make room for other data.
The Hit Ratio of the Single Record Buffer increases very slowly from system startup:
therefore, a hit ratio less than 90% is a concern only if there is no free space left in the buffer.
Low hit ratios are not always due to a performance problem, as an example, a transport into a
system can reduce hit ratios.
Transaction ST04 reports detailed database usage statistics. The Technical team will
watch the buffer, recursive calls, parsed calls, and sorting on this screen. Recursive calls are a
result of low DD (Data Dictionary) cache and parsed calls are a result of low library cache,
both indicate deficiencies in the Shared Pool size.
CPU Management
While monitoring the CPU, the following guideline needs to be followed:
To monitor the operating system performance during stress testing, the UNIX utility
“vmstat” needs to be used. This utility should be set to record data at a set interval (30
seconds) and send it to a binary file that can be analyzed after the test.
.
Work Process CPU Time:
Transaction SM50 should be used to gather information about CPU times used by
each work process in the application and database servers during the test. To calculate CPU
times during each test, the cumulative CPU time of each work process before the test was
subtracted from the cumulative time after the test (shown in ss:hh = seconds, hundredths of
seconds).
The CPU usage of work processes gives an idea of the proper number of work
processes a server needs. If all the work processes of a particular type (like all the dialog
work processes) use large amounts of CPU time, more work processes of that type are
possibly needed. Conversely, if some work processes have a very small change in CPU time,
they can possibly be removed from the configuration.
7. SAMPLE GRAPHS
7.1 UNIX RESOURCE GRAPH
Below are the sample UNIX performance graphs for your reference.
Below is the sample report for the batch jobs execution from SAP system
8. APPENDIX
• IBM P5 m/c Intel CPU 2.8GHz, 2GB RAM, 80GB HDD with MS Widows
2003 Release 2 Standard Edition, Service Pack 1
Software Requirements
• Windows 2003 (Standard and Enterprise editions), this is required if you need
to run more SAP GUI users on a single load injector
• To install LoadRunner on a Windows platform, you must log in as a Local
Administrator.
• Requires Internet Explorer 6.0 SP1 or later.
• If you are running McAfee or Aladdin's eSafe anti-virus applications, close
them before installing LoadRunner.
Tcodes Function
/n Exits the current R/3 screen and displaes initial screen
Sends the current user session to the background
/o and creates a new user session to display the initial screen
/h Turns the debug mode ON
AL08 Displays the list of all users logged on
AL11 Displays SAP directories
SM51 List of servers
Please note that following consideration should be given for the quick correlation in
SAP web which will save much time for the team to identify the correlations..
JessionID
Sap_ext_sid
Sap_wd_tstamp
uwlSessionID
//(J2EE413382100)ID0658168150DB5168491f44203a1890a55a7862908a86b43764dbEnd is
replaced with parameter {C_jsessionid}.
web_reg_save_param( "C_jsessionid", "LB=jsessionid=", "RB=\"", "Ord=1",
"Search=Body", LAST );
• For all the screen shots Loadrunner tool & Loadrunner online references
• sap website for business process steps
E. REFERENCES
• For all the screen shots Loadrunner tool & Loadrunner online references
• Sap website for business process steps