People Tools Performance Guidelines White Paper

An Oracle White Paper 10-September-2010
PeopleTools Performance Guidelines
Table of Contents
TABLE OF CONTENTS ............................................................................................................................................................. 2 CHAPTER 1 - INTRODUCTION .............................................................................................................................................. 6 Structure of this white Paper Related Materials 6 6
CHAPTER 2 - APPLICATION SERVER GUIDELINES ...................................................................................................... 8 Application Server Memory Guidelines 8 Determining PeopleSoft AppServer Memory Usage Under NT................................................................................................. 8 Determining If Excessive Memory Swapping Happens Under NT ............................................................................................ 9 Determining Memory Usage on UNIX ...................................................................................................................................... 9 Determining If Excessive Memory Swapping Occurs Under UNIX ........................................................................................ 10 Application Server Recycle Count 10 Background .............................................................................................................................................................................. 10 Recommendation ...................................................................................................................................................................... 10 Application Server Dynamic Recycle (PeopleTools 8.48 and Later) Shared Cache for Application Server Preload Cache (PeopleTools 8.48 and Later) PSAPPSRV Instances Use Parallel Boot Set Application Server JVM Options Confirm That the Required Bea Tuxedo Version and Minimum Rolling Patch Are Installed 11 11 13 13 14 14 14
CHAPTER 3 - WEB SERVER GUIDELINES ....................................................................................................................... 15 Confirm That the Required Web Server Version, Service Pack, and JRE Version Are Installed Use Jolt Session Pooling (PeopleTools 8.48 and Later) Use Strict Failover and Weighted Load-Balancing (PeopleTools 8.48 and Later) 15 15 15
Oracle Application Server (Oracle AS) 16 Set JVM Heap Size to 256MB or Higher ................................................................................................................................. 16 Capture JOLT Request Timing Traces ..................................................................................................................................... 16 Modify Apache KeepAlive Setting .......................................................................................................................................... 16 Modify ThreadsPerChild and Oc4jCacheSize .......................................................................................................................... 17 Disable HTTP Request Logging in mod_oc4j ......................................................................................................................... 17 Generate Heap Dumps on OutOfMemoryError ........................................................................................................................ 17 WebLogic 17 Log level Setting (PeopleTools 8.49 and Later) ....................................................................................................................... 18 Confirm That the Posix Performance Pack is Loaded .............................................................................................................. 18 Set Thread Count ...................................................................................................................................................................... 19 Confirm JRE Version ............................................................................................................................................................... 19 Set JVM Heap Size to 256MB or Higher ................................................................................................................................. 19
PeopleTools Performance Guidelines White Paper 9/10/2010 Set OS File Descriptor to 100*ThreadCount ............................................................................................................................ 19 Lower OS TCP/IP Cleanup/Timeout Settings .......................................................................................................................... 20 Monitor JVM Garbage Collection ............................................................................................................................................ 20 Disable Servlet Reload ............................................................................................................................................................. 20 Capture JOLT Request Timing Traces ..................................................................................................................................... 21 WebSphere 21 Log Level Setting (PeopleTools 8.49 and Later) ..................................................................................................................... 21 Ensure That JIT is Enabled and Class Verification is Skipped ................................................................................................ 22 Setup and Configuration ........................................................................................................................................................... 22 Set Thread Count ...................................................................................................................................................................... 22 Set JVM Heap Size to 256MB or Higher and Monitor JVM Garbage Collection ................................................................... 22 Other JVM Options .................................................................................................................................................................. 23 Disable Servlet Reload ............................................................................................................................................................. 23 Capture JOLT Request Timing Trace ...................................................................................................................................... 24 Using Resource Analyzer ......................................................................................................................................................... 25 CHAPTER 4 - WEB BROWSER CONFIGURATION ......................................................................................................... 26 Microsoft Internet Explorer ...................................................................................................................................................... 26 Netscape Browser ..................................................................................................................................................................... 26 HTTP 1.1 Compliant Web Browser ......................................................................................................................................... 26 CHAPTER 5 - ADDITIONAL CONFIGURATIONS............................................................................................................ 28 Browser Compression Caching Navigation Pages Caching HTTP KeepAlive 28 28 29 30
Reducing TCP Wait Time 30 For Windows NT ...................................................................................................................................................................... 30 For AIX .................................................................................................................................................................................... 30 For HP/UX 11 or Solaris 2.8 and Later .................................................................................................................................... 31 For Solaris Prior to 2.8 ............................................................................................................................................................. 31 AIX Thread Model Timeout Settings 31 31
CHAPTER 6 - INTEGRATION BROKER ............................................................................................................................ 34 Configure PSBRKHND Message Broker Handler Optimize Message Queue Versus Message Consumption Use of Message Segments for Oracle Delivered Full Sync Service Operations Optimize One Message Versus Multiple Messages per Queue 34 34 34 35
Tuning PSADMIN Parameters for Asynchronous Messaging 35 DISPATCHER Parameters....................................................................................................................................................... 35 PSPUBDSP only ...................................................................................................................................................................... 36 HANDLER Parameters ............................................................................................................................................................ 36 PSPUBHND only ..................................................................................................................................................................... 36 Tuning PSADMIN Parameters for Synchronous Messaging Configure Dedicated Message Servers for High Volume Asynchronous Messaging Configure Dedicated Multiple Domains for High-Volume Asynchronous Messaging 36 37 37
Copyright Oracle. All rights reserved.
PeopleTools Performance Guidelines White Paper
9/10/2010
Application Guidelines for Asynch Messaging 38 Application Guidelines for Sync Messaging ............................................................................................................................ 38 Configure Load Balance Interval (PeopleTools 8.48 and Later) Master/Slave Reducing Maximum App Message Size MultiThreaded Integration Broker (PeopleTools 8.46 and Later) Scan Time Setting for App Messaging PUBSUB Error And App Server Log File Growing 39 39 39 39 40 40
CHAPTER 7 - OPERATING SYSTEM SETTINGS ............................................................................................................. 41 For Linux 41 TCP_WAIT .............................................................................................................................................................................. 41 For Solaris For HP-UX For Tru64 41 42 42
CHAPTER 8 - CALL TELEPHONY INTERFACE (CTI) ................................................................................................... 44 RENServer Parameters (psrenconfig.txt) PSMCAPI Parameters (Renclient.Properties) PSMCAPI Java Options (StartServer.bat) 44 44 45
CHAPTER 9 - PSAE AND PSAESRV .................................................................................................................................... 46 CHAPTER 10 - REPORTING TOOLS .................................................................................................................................. 47 XMLPublisher (PeopleTools 8.48 and Later) Business Objects Enterprise (PeopleTools 8.48 and Later) 47 47
CHAPTER 11 - MONITORING TOOLS ............................................................................................................................... 48 PeopleSoft Ping ........................................................................................................................................................................ 48 psadmin .................................................................................................................................................................................... 48 PeopleSoft Performance Monitor ............................................................................................................................................. 48 CHAPTER 12 - VERSION 8.50 GUI ENHANCEMENTS.................................................................................................... 49 Version 8.50 Enhancements ..................................................................................................................................................... 49 Autocomplete (aka Type Ahead) .............................................................................................................................................. 49 Swan Look and Feel ................................................................................................................................................................. 49 Partial Page Refresh (aka Ajax Updates) ................................................................................................................................. 49 Modal Windows ....................................................................................................................................................................... 49 Scrollable Grids / Grids Zoom ................................................................................................................................................. 50 Mouse Over Pages .................................................................................................................................................................... 50 Deferred Mode vs. Interactive Mode........................................................................................................................................ 50 CHAPTER 13 - QUERY ACCESS SERVICE (QAS) ............................................................................................................ 51 QAS .......................................................................................................................................................................................... 51 Synchronous Query Execution ................................................................................................................................................. 51
PeopleTools Performance Guidelines White Paper 9/10/2010 Asynchronous Query Execution ............................................................................................................................................... 51 Sync/Poll Query Execution ...................................................................................................................................................... 52 Performance Considerations ..................................................................................................................................................... 52 Configuration Recommendations ............................................................................................................................................. 53 APPENDIX B REVISION HISTORY ................................................................................................................................... 54 Authors ..................................................................................................................................................................................... 54 Reviewers ................................................................................................................................................................................. 54 Revision History ....................................................................................................................................................................... 54
PeopleTools Performance Guidelines White Paper 9/10/2010
Chapter 1 - Introduction
This white paper is a practical guide for technical users, installers, system administrators, and programmers who implement, maintain, or develop applications for Oracles PeopleSoft Enterprise Applications. In this white paper, we provide guidelines on how to diagnose a PeopleSoft Online Transaction environment, including PeopleSoft Internet Architecture and Portal configuration. This document does not cover configuration of batch processes. Much of the information contained in this document originated within the PeopleSoft Global Support Center and is therefore based on "real-life" problems encountered in the field. Although this document does not address every conceivable problem that you could encounter with Tuxedo, the PeopleSoft Application Server, or your web server, the issues that appear here are the problems that prove to be the most common or troublesome.
STRUCTURE OF THIS WHITE PAPER

This white paper provides guidance across the major PeopleSoft Internet Architecture components: Application Server, Web Server, Web Browser, Integration Broker, and OS Kernel. Keep in mind that PeopleSoft updates this document as needed so that it reflects the most current feedback that we receive from the field. Therefore, the structure, headings, content, and length of this document are likely to vary with each posted version. To see if the document has been updated since you last downloaded it, compare the date of your version to the date of the version posted on My Oracle Support.
RELATED MATERIALS
This white paper is not a general introduction to environment tuning, and we assume that our readers are experienced IT professionals with a good understanding of PeopleSoft Internet Architecture. To take full advantage of the information covered in this document, we recommend that you have a basic understanding of system administration, basic Internet architecture, relational database concepts/SQL, and how to use PeopleSoft applications. This document is not intended to replace the documentation delivered with the PeopleTools 8 or 8.4x PeopleBooks. To ensure that you have a well-rounded understanding of our PeopleSoft Internet Architecture technology, we recommend that before you read this document, you read information related to PeopleSoft Internet Architecture that is found in the PeopleTools PeopleBooks. Please note that much of the information in this document eventually gets incorporated into subsequent versions of the PeopleBooks. Many of the fundamental concepts related to PeopleSoft Internet Architecture are discussed in the following PeopleSoft PeopleBooks: PeopleSoft Internet Architecture Administration (PeopleTools|Administration Tools|PeopleSoft Internet Architecture Administration) Application Designer (Development Tools|Application Designer) Integration Broker (Integration Tools|Application Messaging) PeopleCode (Development Tools|PeopleCode Reference) PeopleSoft Installation and Administration PeopleSoft Hardware and Software Requirements
PeopleTools Performance Guidelines White Paper 9/10/2010 Additionally, we recommend that you to read documentation on Oracle Application Server, BEA Weblogic/Tuxedo, and IBM Websphere. See your PeopleSoft Installation and Administration PeopleBooks for directions on accessing the Oracle, BEA, and IBM documentation. This white paper has not been submitted for any formal PeopleSoft testing process and has not undergone rigorous review. The material here is published as is. Oracle assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a customer responsibility and depends upon the customer's ability to evaluate and integrate these techniques into their operational environments.
9/10/2010
Chapter 2 - Application Server Guidelines APPLICATION SERVER MEMORY GUIDELINES

Lack of memory resources on the application is the most common cause of serious online performance issues. If you are experiencing more than 4 seconds of response time for every (as opposed to just a few transactions) web page refresh, application server memory is probably to blame. Memory swapping is the most likely culprit. You may experience these same problems when trying to run Microsoft PowerPoint, Word, and Excel on a PC with 12 MB of memory. Here are some tips that you can do to see if you need more memory for your application server: 1. Ensure that you don't have too many PSAPPSRV processes booted. During peak usage times, you should see some small amount of queuing for the psappsrv processes. If you have too many processes, performance is compromised. The extra PSAPPSRV processes not only take up memory space and CPU time from the OS, but also each PSAPPSRV is not sufficiently cached (that is, each PSAPPSRV caches only a portion of the user panels). Fully caching each PSAPPSRV takes more time. To determine the queue length of the specific domain, perform the following steps: a) Run psadmin. b) Go to PeopleSoft Domain Administration for the specific domain name. c) Select TUXEDO command line.
d) Run the command pq. The # Queued column displays the queue length for that application server process, 2. Follow this guideline for the memory footprint for the PSAPPSRV: CRM and HRMS use about 100-150 MB per application server process (CRM gets 50 users per process; HRMS gets less). Supply Chain uses 300-500 MB per process (you get about 10-20 users per process). Financials uses 150-300 MB per process (you get about 20-30 users per process). So, for example, if you have 1GB of memory available for a financials application server, boot no more than four PSAPPSRV processes. 3. Ensure that you include all test, development, and production domains on a computer in your calculation. A common problem is having four large domains on one computer with insufficient memory. 4. Check your memory utilization on the application server when you experience performance problems. Check for swapping by using the memory monitoring procedures outlined in the following sections. 5. If you are swapping, either add more memory or reduce the number of domains or PSAPPSRV processes on the computer.
Determining PeopleSoft AppServer Memory Usage Under NT

To determine PeopleSoft Application Server memory usage under NT: 1. Use NT TaskManager to monitor all the PeopleSoft processes.
PeopleTools Performance Guidelines White Paper 9/10/2010 PeopleSoft process names begin with ps. 2. Start TaskManager and select View from the menu bar, and then select Select Columns. Pick the Memory Usage and Virtual Memory Size columns. 3. Sort the process by name, and the two memory columns will tell you approximately the amount of memory that each PeopleSoft process is using. Memory Usage is not the entire memory consumption of the process, but only the amount of physical RAM that the process is using. However, with NT, an application may be using much more Virtual Memory (that is, the paging file) than physical memory, so if you're gauging memory at all, you must select both of these fields. The Memory Usage columns includes the shared libraries loaded into each process; therefore, if you add up all the processes, you will be double counting the shared libraries portion. Because PeopleSoft shares a lot of the common libraries, the summation of all PeopleSoft processes is a quick estimator, not an absolute measurement of memory usage. The total Memory Usage and the VM Size of the combined PeopleSoft processes should not be greater than 70 percent of the real memory of the server.
Determining If Excessive Memory Swapping Happens Under NT

To determine if excessive memory swapping happens under NT: 1. 2. Start the NT Performance Monitor, perfmon. Add the Object Counter, under Memory, and then the Page/sec. Pages/sec is the number of pages read from the disk or written to the disk to resolve memory references to pages that were not in memory at the time of the reference. This is the sum of Pages Input/sec and Pages Output/sec. This counter includes paging traffic on behalf of the system cache to access file data for applications. This value also includes the pages to/from noncached mapped memory files. This is the primary counter to observe if you are concerned about excessive memory pressure (that is, thrashing) and the excessive paging that may result. When page/sec is greater than 100 hard pages per second, there is a swapping problem. Reduce the number of PSAPPSRV processes or add more memory to the server.
Determining Memory Usage on UNIX

To help the end user better understand the memory consumption and CPU usage of the PeopleSoft Application Domain, we have created several UNIX vendor-specific shell scripts to monitor the Performance Monitor parameters. You can obtain the monitoring shell scripts from My Oracle Support document id 333300.1 Here is a sample output of the ps_chk_domain script: Please Enter your application server domain name? PT84 Please Enter interval in seconds? 10 Please Enter # of interval's? 100 Tue Oct 23 23:00:33 PDT 2001 For all PS Processes: Resident memory size is: CPU for PS Processes: 123.543 MB 11 %
PeopleTools Performance Guidelines White Paper Virtual memory size is:
9/10/2010 198.098MB
For all PSAPPSRV Processes: Resident memory size is: 108.453MB CPU for PS Processes: 9% Virtual memory size is: 106.559 MB Resident memory refers to the real physical memory currently required by the process for its operation. Virtual memory refers to the process virtual address size, which includes memory that has been paged out to the physical disk. If the virtual memory continues to increase, you should lower the RecycleCount of the AppServer. The output of ps_chk_domain.sh divides the portion of PSAPPSRV processes from the entire domain so that the end-user (or system administrator) can get a better understanding with the distribution of the memory resources. We recommend that the total resident memory for the entire PS Processes not exceed 70 percent of the total real memory available on the server.
Determining If Excessive Memory Swapping Occurs Under UNIX

To determine if excessive memory swapping occurs under UNIX, use vmstat to determine if the OS is swapping. To check whether paging is a problem, run vmstat and check out the pi column to see if you are doing much paging in and po for paging out. You can also run sar -q to check paging or run vmstat -s and take multiple samples to see if the page ins and outs to paging space are the majority of the total paging.
APPLICATION SERVER RECYCLE COUNT

The Application server recycle count parameter dictates the number of services after which the AppServer automatically restarts.
Background
The AppServer processes use physical runtime memory to cache Panel objects in order to speed up user response time, instead of fetching the Panel objects from the database server every time. However, as the number of requests serviced by the AppServer increases, the amount of physical memory occupied by the AppServer processes also increases. When the amount of memory occupied by the AppServer becomes too large (relative to the real memory available at the time), paging to a file system occurs and impacts user experience. In order to effectively manage the memory footprint of the AppServer, keep the Recycle Count at a realistic level. When the AppServer reaches the specified Recycle Count value, the AppServer terminates and restarts itself. When the AppServer terminates, the occupied memory is released.
Recommendation
We recommend that you set the recycle count to 5,000. Adjust the recycle count so that no memory swapping is introduced because of a high recycle count value. (See the previous session for information how to determine memory swapping.)
10
PeopleTools Performance Guidelines White Paper 9/10/2010 To minimize the cost of recaching the AppServer, you must enable File Cache on the AppServer. To enable Server Caching, set the EnableServerCaching=2 (the default value is 2). ----------------------------------------------------------------------; EnableServerCaching ; 0 Server File caching disabled ; 1 Server File caching limited to most used classes ; 2 Server File caching for all types EnableServerCaching=2 -----------------------------------------------------------------------
APPLICATION SERVER DYNAMIC RECYCLE (PEOPLETOOLS 8.48 AND LATER)

Application server restarts can cause inconsistencies in response times. If you set the recycle count too low, more process restarts will occur. If you set the recycle count too high, the processes can grow big, resulting in reduced response times due to swapping or increased likelihood of crashing due to memory violation. PeopleTools 8.48 and later provides a new feature to recycle an app server process when its memory growth exceeds a configurable threshold. When you enable this feature, the application server will service (at a minimum) the number of requests specified by the recycle count. The application server will continue to accept requests beyond the recycle count until its memory growth (that is not due to metadata loads) exceeds the configurable threshold. Note that if an app server exceeds the threshold before reaching the recycle count, then the system logs a delaying recycle message. To use this feature, consult the PeopleBook: System and Server Administration under PSAPPSRV Options. Customers who are interested in this feature should initially experiment with it in a staging environment to determine an appropriate threshold for their specific applications and environments before using the feature in production. Optionally, customers can use the recycle count achieved using the memory growth option in the staging environment as a static recycle count in the production environment.
SHARED CACHE FOR APPLICATION SERVER

PeopleTools 8.4x has a shared cache feature in which the various psappsrv processes in an app server domain use a single set of cache files rather than separate cache files for each process. All managed objects are included in the shared cache files so that objects are not loaded from the database. Note: You must preload all the necessary cache objects before you can enable the Shared Cache feature. PeopleSoft installation is supplied with an Application Engine program called LOADCACHE to help customers generate preloaded shared cache. Shared cache provides the following benefits: Saves disk space because all AppServer processes use the same common directory to read the file cache contents. Speeds up operation because all managed objects are cached into the common directory ahead of time, instead of retrieving from the database tables on demand.
To run the LOADCACHE program: 1. Ensure that the database that the application server runs against produces clean SYSAUDIT runs. If SYSAUDIT is not clean, the LOADCACHE program may fail. 2. Ensure that the user who is running the program has defined the Load Application Server Cache component to the Permission List in User Profile. 11
9/10/2010
Find or add the Utilities page for the users permission list and add the Load Application Server Cache component. 3. (For PeopleTools 8.42 and earlier only) Check your psprcs.cfg file (Process Scheduler configuration file) and set it appropriately. psprcs.cfg is where you specify the type of objects to cache using the EnableServerCaching parameter. For PeopleTools 8.4x, set it to 2. The LOADCACHE reads this setting and caches metadata according to the value specified in the Process Scheduler configuration. However, do not enable shared caching for Process Scheduler. Ensure that ServerCacheMode equals 0. [Cache Settings] ;============================================ ; Settings for Tools that use Cache ;============================================ CacheBaseDir=%PS_SERVDIR%\CACHE ;----------------------------------------------------------------------; EnableServerCaching ;0 Server File caching disabled ;1 Server File caching limited to most used classes ;2 Server File caching for all types EnableServerCaching=2 ;----------------------------------------------------------------------; CacheBaseDir = the base cache directory ;CacheBaseDir=%PS_SERVDIR%\CACHE ;----------------------------------------------------------------------; ServerCacheMode ;0 One cache directory per App Server Process ;1 Shared Cache ServerCacheMode=0 3. 4. (For PeopleTools 8.40 and later) Select PeopleTools, Utilities, Administration, Load Application Server Cache. Enter the appropriate Run Control ID. You may have to add a new value here. The Load Application Server Cache page appears. 5. In the Output Directory, specify the directory where you want the cached metadata to be written: Unix example> /ds1/home/testora/pt814c1/PT814U25/appserv NT example> c:\temp\ 6. Select the correct process scheduler and click Run. Navigate to PeopleTools>Process Monitor. Wait for the process to become Success. The first time that you run the process, it may take 4 to 5 hours. Ensure that the Run Location is set to Server. 7. 8. Shut down your application server domain. Enable shared caching with the ServerCacheMode parameter (ServerCacheMode=1) in psappsrv.cfg of the domain, and reconfigure the domain so that the changes are reflected. [Cache Settings] ;============================================ ; Settings for Tools that use Cache ;============================================ CacheBaseDir=%PS_SERVDIR%\CACHE ;----------------------------------------------------------------------; EnableServerCaching ;0 Server File caching disabled ;1 Server File caching limited to most used classes ;2 Server File caching for all types EnableServerCaching=2
12
PeopleTools Performance Guidelines White Paper 9/10/2010 ;----------------------------------------------------------------------; CacheBaseDir = the base cache directory ;CacheBaseDir=%PS_SERVDIR%\CACHE ;----------------------------------------------------------------------; ServerCacheMode ;0 One cache directory per App Server Process ;1 Shared Cache ServerCacheMode=1 9. Create the <PS_HOME>\<DomainName>\cache\share directory for the appropriate domain.
10. Navigate back to your Process Scheduler. You will now have some cache generated under a new stage directory. Unix example: /ds1/home/testora/pt814c1/PT814U25/appserv/CACHE/stage NT example: c:\temp\CACHE\stage 11. When you see a bunch of files with *.dat and *.key extensions inside the stage directory, copy the contents of the stage directory into the share directory on your appserver domain. 12. Reboot your application server domain. Note: For shared cache users: shared cache is intended to be used in a disciplined environment. If you use shared cache, do not make any application changes that will affect the Meta Data objects, such as the Panel definition, Menu structure, or Permission List. The reason: once you chose to use shared cache, the cache files are all marked as read only. The AppServer processes cannot update the cache files for any delta, and each access to the changed Meta Data will result in a costly database access. The preferred way to use LoadCache files is to copy the entire cache file set to the individual AppServer process (that is, seeding the cache directory with the prebuilt LoadCache fileset). Multiple copies of the cache file in each directory consume disk space, but enable the process to update the cache file content. This flexibility enables you to make application changes but does not affect performance.
PRELOAD CACHE (PEOPLETOOLS 8.48 AND LATER)

To improve performance, the application server uses a caching mechanism that keeps commonly used objects in memory or file form on the application server to reduce the need for a database request each time that you access a component or page. As you access more pages and components, more data is stored in the application server cache. However, if you have not already accessed a page, for example, that page wont exist in the current cache, and you may experience a slower response time as the system requests the page from the database. To prevent this initial performance degradation, in PeopleTools 8.48 and later, you can elect to preload files or memory cache with commonly used components. Preloading cache involves creating a project containing commonly used components and then referring to these projects in the PSADMIN settings PreloadMemoryCache and PreloadFileCache. To understand and use this feature, see the PeopleBook: System and Server Administration under the section Configuring an Application Server Domain to Preload Cache.
PSAPPSRV INSTANCES
Keep the number of PSAPPSRV instances low. PSAPPSRV must cache PeopleSoft meta-objects in order to be effective. Too many PSAPPSRV process instances make it difficult to fully cache all of them. Too many process instances become even more troublesome in a Win2000/NT Tuxedo domain because the Bulletin Board process tries to use the same PSAPPSRV process ID (not using the round robin method as in UNIX).
13
9/10/2010
We recommend starting with 1.5 to 3 PSAPPSRV processes per CPU. If the application server machine is 100 percent utilized under load, reduce the number of PSAPPSRV processes until there is some idle CPU. If as a result, there is significant queuing, then you may need additional application server hardware. In general, it is best to allow a small amount of queuing during peak working hours. If the app server machine is more than 80 percent loaded, then we do not recommend spawning an extra PSAPPSRV instance to handle high-volume workload. Every time that a new PSAPPSRV process starts, the process must establish its cache, which takes time, CPU, and memory. Use an appropriate "min" setting to indicate how many PSAPPSRV processes are required for proper throughput. However, if the app server machine has sufficient headroom for better utilization, then you can use spawning to avoid excessive queuing. To disable spawning, use the same value in the Min Instances and Max Instances fields.
USE PARALLEL BOOT

Prior to PeopleTools 8.48, server processes in a domain started serially when the domain was booted. PeopleTools 8.48 and later provide an option for parallel booting. We recommend using parallel booting to significantly speed up the process of starting a domain. Note: In PeopleTools 8.48, when PeopleSoft Performance Monitor is enabled, parallel booting may not be effective on Linux, AIX, HP/UX, and Tru64 due to limitations in Java 1.4.2 libraries. A potential workaround is to set the following application server JVM option: -Djava.security.egd=file:/dev/urandom
SET APPLICATION SERVER JVM OPTIONS

Customers using functionality that does a lot of processing in Java (for example, XMLPublisher) should consider setting the JVM options for the application server. The default JVM options may not be optimal. In that case, we recommend adding the following setting to psappsrv.cfg or psprcs.cfg (in addition to the already existing options): -server -Xms128m -Xmx512m
CONFIRM THAT THE REQUIRED BEA TUXEDO VERSION AND MINIMUM ROLLING PATCH ARE INSTALLED
Confirm that the required Bea Tuxedo version and the minimum rolling patch are installed specific to the PeopleTools release and operating system from Certifications section of My Oracle Support. You can verify the installed Bea Tuxedo rolling patch from the following rolling patch level file: <TUXDIR>\udataobj\patchlev
14
Chapter 3 - Web Server Guidelines CONFIRM THAT THE REQUIRED WEB SERVER VERSION, SERVICE PACK, AND JRE VERSION ARE INSTALLED
Confirm that the required web server version, service pack, and JRE version are installed specific to the PeopleTools release and operating system from the Certifications section of My Oracle Support.
USE JOLT SESSION POOLING (PEOPLETOOLS 8.48 AND LATER)

In PeopleTools 8.48 and later, Jolt Session Pooling is enabled by default. Without Jolt Session Pooling, each user session requires a dedicated Jolt session between web server and app server, which consumes valuable system resources. With Jolt Session Pooling, the Jolt sessions are shared among users, which reduces system resource usage. In our intenal test on a 2-CPU Windows box, we were able to run 500 users with Jolt Session Pooling versus only 280 users without Jolt Session Pooling. To check the Jolt Session Pooling setting, select web.xml under the appropriate directory for each web server platform. To verify if Jolt Session Pooling is enabled, check the web.xml file: <servlet> <servlet-name>psc</servlet-name> <servlet-class>psft.pt8.psc</servlet-class> . . . <init-param> ... <param-name>joltPooling</param-name> <param-value>true</param-value>
USE STRICT FAILOVER AND WEIGHTED LOAD-BALANCING (PEOPLETOOLS 8.48

AND LATER)
Prior to PeopleTools 8.48, when a host failed, the load was redirected to the next host on the server list. Redirection caused system load to increase significantly on that host and made the load unstable. With PeopleTools 8.48 and later, you can specify a failover target for each host and thus distribute the failover load. You can also specify a weight for each host so that more load is directed to more powerful machines. To configure strict failover and weighted load balancing, edit psserver property in configuration.properties. For example: psserver=appserver1:9000#3[appserver99:10010],appserver2:9010#1[appserver98:10010] In this case, appserver1 would receive three times more requests than appserver2. If appserver1 fails, the requests will be routed to appserver99. The load may not always be distributed per the weights, especially when the load is light.
15
9/10/2010
ORACLE APPLICATION SERVER (ORACLE AS)

Set JVM Heap Size to 256MB or Higher
The Java virtual machine heap space is the memory region where the Java objects (both live and dead) resided. When the Java heap runs out of space, the Java Garbage Collector is invoked to deallocate unreferenced objects and free up more space for the program to continue its operation. The JVM cannot service user requests during garbage collections. Many customers have their JVM heap size set to a minimum heap size of 64MB and maximum size of 256MB. Setting the JVM heap size to a greater minimum value (preferably equal to the maximum value) avoids the performance hit incurred by dynamically growing the JVM and improves predictability; it also lessens the frequency of JVM garbage collection. To set the heap size for PeopleSoft Internet Architecture, open opmn.xml in <ORACLE_HOME>\opmn\conf for editing. Locate the process-type node for your PeopleSoft Internet Architecture installation and add this line to the value attribute of the java-options node: -Xms256m Xmx256m verbosegc The verbosegc switch enables you to monitor the amount of heap usage and the time Oracle AS took for garbage collections so that you can make further adjustments if necessary. The garbage collection details are written to the log file <ORACLE_HOME>\opmn\logs\OC4J~<PIA Install Name>~default_island~<jvm instance number>. Depending on which applications you are using, you may need to set the heap size even higher.
Capture JOLT Request Timing Traces

To collect JOLT request timing traces for an Oracle AS installation of PeopleSoft Internet Architecture: 1. Using the Custom Properties page of the current WebProfile, add a String property named auditPWD and set the value. You will use this password in a later step. 2. 3. 4. 5. Stop the webserver. Open opmn.xml in <ORACLE_HOME>\opmn\conf for editing, locate the process type node for your PeopleSoft Internet Architecture installation, and add -Dloggersize=0 to the value attribute of the java-options node. Restart the webserver. Before logging on, submit the following URL to reset the existing log: http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1>. A screen of messages appears on the browser window. That's the log details that has been collected. 6. 7. 8. Point to the logon URL and logon as usual. After logging on, point to the previously mentioned URL again (http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1>) to retrieve the log. Copy and paste the log from the browser to a text file.
For more comprehensive data, repeat steps 4 through 7.
Modify Apache KeepAlive Setting

Depending on the expected interval between your users requests, it may be beneficial to change the Apache KeepAlive Timeout from 15 to 30 seconds and set the Maximum KeepAlive Requests to zero. Open <ORACLE_HOME>\Apache\Apache\conf\httpd.conf for editing. Locate the KeepAlive, MaxKeepAliveRequests, and KeepAliveTimeout lines and set them to On, 0, and 30, respectively. Because KeepAlive controls how long OHS retain the connection between the client and OHS, it is possible that if there are a large number of concurrent users, increasing KeepAlive
16
PeopleTools Performance Guidelines White Paper 9/10/2010 from 15 to 30 will negatively impact performance because there will be fewer available connections to service the requests. You should change this setting only if you see excessive CPU consumption of the OHS process (look for apache.exe in your Task Manager in Windows or do ps ef | grep httpd in UNIX) and you expect a stable number of users making requests at interval to be longer than the default setting of 15 seconds. When you increase KeepAlive, you may need to increase MaxClients or ThreadsPerChild (see the next section) to ensure incoming requests are not starving for connections.
Modify ThreadsPerChild and Oc4jCacheSize

For Windows, the ThreadsPerChild setting in <ORACLE_HOME>\Apache\Apache\conf\httpd.conf controls the number of concurrent threads (requests) that the Oracle HTTP Server can service. In most cases, the default value of 150 is sufficient. For UNIX, the equivalent setting is MaxClients. In most cases, the default value of 150 is sufficient. For Windows, if you decide to increase ThreadsPerChild, you should set the Oc4jCacheSize to the same value. This setting controls the number of connections between the Oracle HTTP Server and OC4J. Setting the Oc4jCacheSize to the same value as ThreadsPerChild should provide the optimal performance. To change this value, open <ORACLE_HOME>\Apache\Apache\conf\mod_oc4j.conf and search for Oc4jCacheSize. If the entry is not there, add the following line above the first Oc4jMount entry (this assumes that you have increased ThreadsPerChild to 150): Oc4jCacheSize 150 You do not need to change this setting for UNIX.
Disable HTTP Request Logging in mod_oc4j

Open mod_oc4j.conf in <Apache_home>\conf for editing. Locate the appropriate <virtual host> section and comment out the line beginning with TransferLog.
Generate Heap Dumps on OutOfMemoryError

If you are using JDK 1.4.2_12 or later, there is a new option to dump the JVM heap if you are running into OutOfMemoryError. You can add this comment to the JVM options: XX:+HeapDumpOnOutOfMemoryError XX:HeapDumpPath=/tmp If there is an OutOfMemoryError, the contents of the heap will be dumped to a file in the directory specified by: XX:HeapDumpPath The heap dump file can be read by the Heap Analysis Tool (HAT), jhat (in JDK 6), or a profiler that supports the binary heap dump format. The heap dump file provides valuable information to Oracle development on whats causing the OutOfMemoryError.
WEBLOGIC
17
9/10/2010
Log level Setting (PeopleTools 8.49 and Later)

In Weblogic 9.2, log file severity level has been set to Info by default. The severity level produces too much logging information, and the Weblogic server must index all logs that caused performance degradation in internal performance tests. For the server log type, only three options are available for the log severity field: Debug, Info, and Warning. We recommend that you set the log file severity level to Warning and set the log filter to Critical. These settings will improve performance by reducing the overhead of log indexing, rotation, and so on. Log severity levels and log filters are independent of one another. This means setting log filters is not mandatory. In order to set the log severity (for server log) to Critical, you must use the filter because the log severity field does not provide Critical as one of the options. To improve performance, you can set the log severity of three other types of logs (Standard out, Memory buffer, and Domain log Broadcaster) to Critical without using a filter. We recommend that you select the Redirect stdout logging enabled option. When you enable this option, the stdout of the JVM in which a WebLogic Server instance runs is redirected to the WebLogic logging system. If you do not select this redirect option, the stdout content is published to the server terminal console and performance is compromised. To configure recommended logging level: a). Log in to the Weblogic Admin console and follow this path to create a filter: i. Domain -> configuration -> Log Filter -> click the Lock& Edit button ->New -> Name=Critical -> Finish ii. Click the Critical link -> Edit -> Enter (SEVERITY=Critical) -> Finish -> Save -> Click Activate changes To configure logging levels: iii. Follow this navigation: Environment ->Servers ->PIA(admin) ->Logging -> Advanced ->click the Lock& Edit button iv. Use these log file settings: 1. Severity level= Warning 2. Filter= Critical 3. Ensure that the Redirect stdout logging enabled check box is enabled. v. Use these standard out settings: 1. Severity level= Critical 2. Filter= Critical vi. Use these domain log broadcaster settings: 1. Severity level= Critical 2. Filter= Critical vii. Use these memory buffer settings: 1. Severity level= Critical 2. Filter= Critical viii. Click Save and then click Activate changes.
Confirm That the Posix Performance Pack is Loaded

Under normal circumstances, WebLogic should use the Posix Performance Pack, and this Posix Performance Pack will use the native OS's socket implementation. When the Performance Pack is not loaded properly, generic socket implementation is used. However, generic socket implementation is not efficient and could cause performance issues or socket stability problems. To verify that your WebLogic Server is loading the Performance Pack correctly, check your WebLogic output message and search for the reference to NT/Posix Performance Pack.
18
Set Thread Count

In WebLogic 9.2 onwards, you do not set the thread count. threads are automatically allocated as needed. For more information, see My Oracle Support Document Id 660080.1. For WebLogic 8.1, please refer to My Oracle Support Document Id 638118.1 for instructions to configure the Thread Count. Note: For Unix platforms, when you increase the ThreadCount, you are allowing more socket connections to be established. Therefore, you must increase the number of file descriptors (maxfiles and maxfiles_lim) accordingly. To check the current file descriptor value, use the following command: csh c limit h descriptors You must also raise the number of threads limit per process (max_thread_proc) in UNIX. As for Windows, these are implicitly limited by other system resources such as memory, and there is no explicit parameter that controls them.
Confirm JRE Version

Confirm from BEA's platform page that the installed JRE is certified (may require OS patches). Here is the path to the BEAs platform page: http://download.oracle.com/docs/cd/E13196_01/platform/suppconfigs/index.html
Note: For Linux, it is necessary to set an environmental variable to work around a JRE bug. $ export J2SE_PREEMPTCLOSE=1 The rationale behind this setting is that the current Linux is still using the old UNIX network/thread semantics, and the close() is not preemptive as in Solaris and AIX. This setting applies to JRE1.2.2, 1.3 and 1.3.1.
Set JVM Heap Size to 256MB or Higher

Many customers have their JVM heap size set to a minimum heap size of 64MB and maximum size of 256MB. Depending on which applications you are using, you may need to set the heap size even higher. Setting the JVM heap size to a larger minimum value (preferably equal to the maximum value) avoids the performance hit incurred by dynamically growing the JVM and improves predictability. Increasing the setting also lessens the frequency of JVM garbage collection. See the following JVM Heap Size section to learn how to change the heap size for a specific web server.
Set OS File Descriptor to 100*ThreadCount

A file descriptor is required for every file that is opened, every *.class file read in by WebLogic, every jolt connection PIA/Portal make to the appserver, every connection that has to open back to a client, and any other socket-based communication that was occurring on that machine. To raise the file descriptors for UNIX, use the following command: ulimit n 4096
19
9/10/2010
In Windows NT/2000, there is not an explicit parameter for the number of file descriptors. The parameter is implicitly limited by hardware resources (mainly system memory).
Lower OS TCP/IP Cleanup/Timeout Settings

Socket-based applications that are opening and closing hundreds or thousands of sockets must have their sockets marked as closed. Once a process closes a socket, it is marked as closed only until the OS (based on a cleanup/flush timeout) makes that socket available again. The default value for this is 11 minutes. Lower this value to 1 minute. Planet PeopleSoft implements a 1-minute TCP wait time. BEA also provides informationo on this topic. See the Reducing TCP Wait Time section to learn how to change the TCP Wait Time for a specific OS.
Monitor JVM Garbage Collection

The Java virtual machine heap space is the memory region where the Java objects (both live and dead) reside. When the Java heap runs out of space, the Java Garbage Collector invokes to deallocate the dead objects and free up more space for the program to continue its operation. The JVM cannot service user requests during garbage collections. To monitor the amount of heap usage and the time that WebLogic takes for the garbage collection, you can add the verbosegc switch to the setEnv.cmd script file. You have to start the WebLogic from the command line -- startPIA.cmd (instead of an NT service) to see the GC output. In setEnv.cmd: SET JAVA_OPTIONS=-hotspot Xms256m Xmx256m verbosegc Here is a sample output of the GC: Sat Nov 24 22:15:34 PST 2001:<I> <WebLogicServer> Invoking garbage collection Sat Nov 24 22:15:34 PST 2001:<I> <GC> GC: Before free/total=46867368/67108856 (69%) <GC: freed 249213 objects, 15440712 bytes in 396 ms, 95% free (51334096/53687088)> <GC: init&scan: 6 ms, scan handles: 105 ms, sweep: 124 ms, compact: 161 ms> <GC: 0 register-marked objects, 140 stack-marked objects> <GC: 1 register-marked handles, 559 stack-marked handles> <GC: refs: soft 0 (age >= 32), weak 0, final 559, phantom 0> <GC: compactHeap: blocks_moved=249506> <GC: 0 explicitly pinned objects, 35 conservatively pinned objects> <GC: last free block at 0x02A11B2C of length 35906768, is at end> To minimize performance degradation from Garbage Collection, use the command line option -noclassgc. This option inhibits a thread that would normally clear out unused classes (thus saving the load incurred by that thread). The goals of tuning your heap size are twofold: minimize the amount of time that you spend doing GC while maximizing the amount of clients that you can handle at a given time.
Disable Servlet Reload

In %WLS_HOME%/<peoplesoft web domain>/config.xml, there is a parameter (ServletReloadCheckSecs) that dictates how often WebLogic checks whether a servlet has been modified, and if so reloads it: <WebAppComponent Name="PORTAL" ServletReloadCheckSecs="-1" Targets="PIA" URI="PORTAL" />
20
PeopleTools Performance Guidelines White Paper 9/10/2010 In a production environment, the servlets are not modified. Therefore, checking and reloading only incurs unnecessary work. Thus, you should set ServletReloadCheckSecs to 1 for each of the components. (If the field is not present, the value defaults to 0 always reload, which is undesirable. If this is the case, introduce the field with a value of 1.)
Capture JOLT Request Timing Traces

To capture JOLT request timing traces: 1. Using the Custom Properties page of the current WebProfile, add a String property named auditPWD and set the value. You will use this password in a later step. 2. 3. Stop the webserver. In the weblogic domain directory, open startPIA.cmd/startPIA.sh and add a command-line option -Dloggersize=0 to the firing of the java process, like the following: %JAVA_HOME%\bin\java" %JAVA_OPTIONS% -classpath %CLASSPATH% Dweblogic.Domain=%DOMAIN_NAME% -Dweblogic.Name=%SERVER_NAME% Dbea.home="%BEA_HOME%" -Dweblogic.management.password=%SYSTEMPASSWORD% Dweblogic.ProductionModeEnabled=%STARTMODE% "Djava.security.policy==%BEA_HOME%/wlserver6.1/lib/weblogic.policy" Dweblogic.management.discover=%DISCOVERY_MODE% -Dloggersize=0 weblogic.Server 4. 5. Restart the webserver Before logging on, submit the following URL to reset the existing log: http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1> A screen of messages appears on the browser window. The messages represent the log that has been collected. 6. 7. 8. Point to the logon URL and logon as usual. After logging on, point to the previous URL again (http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1>) to retrieve the log. Copy and paste the log from the browser to a text file.
WEBSPHERE
Log Level Setting (PeopleTools 8.49 and Later)
In Websphere 6.1.0.3, log detail level has been set as Info by default. This setting produces too much logging data to logfile, which degrades performance in internal performance tests. We recommend changing the log detail level from *=info to *= severe to improve the performance up to 20 percent. Log in to the Websphere admin console and navigate to Troubleshooting> Logs&Trace> server1>Change Log Detail Levels. Set the value to *=severe. We recommend that you disable the diagnostic trace log, which reduces the amount of logs written in webserver log files. Log in to the Websphere admin console and navigate to Troubleshooting > Logs&Trace > server1 > Diagnostic trace. Clear the Enable log check box.
21
9/10/2010
Ensure That JIT is Enabled and Class Verification is Skipped

Ensure that JIT is enabled by default in the Admin console. If you disable the JIT compiler, throughput decreases noticeably. Therefore, for performance reasons, keep JIT enabled. JVM property: -Xverify:none. When using this value, the class verification stage is skipped during class loading. By using Xverify:none with the just in time (JIT) compiler enabled, performance test indicate that startup time is improves up to 15 percent.
Setup and Configuration

Note: PeopleTools 8.4 supports a native http server within WebSphere at port 9080 and 9443, for http and https, respectively. Use of IBM Http Server, which listens at port 80 or 443 for http or https, is optional. Note: Before you begin the installation, review the PeopleSoft Platforms Database to make sure that you are installing WebSphere into a supported environment. For detailed installation instructions, especially the required e-fixes needed for WebSphere, see the WebSphere Install document posted on Customer Connection. Note: Install WebSphere eFixes in any order. To remove eFixes, roll back efixes in the reverse order (from which you applied them).
Set Thread Count

1. Step 1. Locate server.xml at: %PS _HOME%\webserv\{profile_name}\config\cells\{cell_name}\nodes\{node_name}\servers\server1\ and modify the thread pool maximumSize for web container, <services xmi:type="threadpoolmanager:ThreadPoolManager" xmi:id="ThreadPoolManager_1208430577007" enable="true"> ... ... <threadPools xmi:id="ThreadPool_1208430577008" minimumSize="10" maximumSize="50" inactivityTimeout="3500" isGrowable="false" name="WebContainer"/> ... ... </services> 2. Step 2. Login to WebSphere admin console and navigate to: servers -> Application server -> server1 -> Thread Pool -> WebContainer and modify the thread pool maximumSize for the web container.
Set JVM Heap Size to 256MB or Higher and Monitor JVM Garbage Collection
(Refer to the WebLogic section on how to decide the JVM heap size.)
22
PeopleTools Performance Guidelines White Paper 9/10/2010 In %WAS_HOME%/config/server-cfg.xml, locate the following lines: <jvmSettings xmi:id="JavaVirtualMachine_1" classpath="${WAS_ROOT}/lib/bootstrap.jar;${WAS_ROOT}/properties;${WAS_ROOT}/ins talledApps/peoplesoft/PORTAL/WEBINF/lib/entappletbase.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entapplethttp.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entappletp10.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entappletp12.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entappletp5.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entappletp7.jar;${WAS_ROOT}/installedApps/peoplesoft/PORTAL/WEBINF/lib/entappletssl.jar" bootClasspath="" verboseModeClass="false" verboseModeGarbageCollection="false" verboseModeJNI="false" initialHeapSize="256" maximumHeapSize="256" runHProf="false" hprofArguments="" debugMode="false" debugArgs="" genericCommandLineArgs="com.ibm.ws.runtime.StandardServer" disableJIT="false"></jvmSettings> Note that you can also turn on verbosegc here (verboseModeGarbageCollection). With IBM Websphere 5, you can go to the administration console (http://:9090/admin/) to change settings. Log in with a blank user ID. On the left Navigation, select Servers -> Application Servers. Select your server name (default is server1). Click Process Definition under Additional Properties. Click Java Virtual Machine under additional Properties. Here you can change all the parameters where you need verbosegc and so on.
Other JVM Options

Insert other JVM options if needed (for example, noglassgc) in the server-cfg.xml file rather than the startup script (startserver). For example:
<processDefinition xmi:type="server:JavaProcessDef" xmi:id="ProcessDef_1" executableName="${JAVA_HOME}/bin/java" commandLineArguments="-noclassgc" workingDirectory="${WAS_ROOT}/bin" executableTargetKind="JAVA_CLASS" executableTarget="com.ibm.ws.bootstrap.WSLauncher"></processDefinition>
Disable Servlet Reload

The servlet reload parameters in WebSphere is located in %WAS_HOME%/<peoplesoft web domain>/[PORTAL/PSIGW/PSINTERLINKS]/WEB-INF/ibm-web-ext-xmi (usually this is the first line): (In Websphere 5, the xmi file is located here: %WAS_HOME%\AppServer\config\cells\\applications\peoplesoftWAS.ear\deployments\peoplesoftWAS\PSINTERLINKS\W EB-INF.) <webappext:WebAppExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:webappext="webappext.xmi" xmlns:webapplication="webapplication.xmi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmi:id="WebAppExtension_1" reloadInterval="0" reloadingEnabled="false" fileServingEnabled="true" directoryBrowsingEnabled="true" serveServletsByClassnameEnabled="false"> (In the PSINTERLINKS/WEB-INF directory, such file may not exist. If this is the case, create a file called ibm-web-ext.xmi with the following content: <webappext:WebAppExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:webappext="webappext.xmi" xmlns:webapplication="webapplication.xmi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
23
9/10/2010
xmi:id="WebAppExtension_1" reloadInterval="0" reloadingEnabled="false" fileServingEnabled="true" directoryBrowsingEnabled="true" serveServletsByClassnameEnabled="false"> <defaultErrorPage xsi:nil="true"/> <additionalClassPath xsi:nil="true"/> <webApp href="WEB-INF/web.xml#WebApp_1"/> <extendedServlets xmi:id="ServletExtension_1"> <extendedServlet href="WEB-INF/web.xml#Servlet_1"/> </extendedServlets> </webappext:WebAppExtension> Also copy the file ibm-web-bnd.xmi from the PORTAL/WEB-INF directory.) Moreover, %WAS_HOME%/<peoplesoft web domain>/META-INF/ibm-application-ext-xmi should look like this (with the reloadInterval line deleted): <applicationext:ApplicationExtension xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:applicationext="applicationext.xmi" xmlns:application="application.xmi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmi:id="Application_ID_Ext" reloadInterval="0">  <application href="META-INF/application.xml#Application_ID"/> </applicationext:ApplicationExtension> Refer to the same topic in WebLogic for further information.
Capture JOLT Request Timing Trace

To collect JOLT request timing traces for WebSphere: 1. Using the Custom Properties page of the current WebProfile, add a String property named auditPWD and set the value. You will use this password in a later step. 2. 3. Stop the webserver. In %WAS_HOME%/config/server-cfg.xml, locate the following lines: <processDefinition xmi:type="server:JavaProcessDef" xmi:id="ProcessDef_1" executableName="${JAVA_HOME}/bin/java" commandLineArguments="-Dloggersize=0" workingDirectory="${WAS_ROOT}/bin" executableTargetKind="JAVA_CLASS" executableTarget="com.ibm.ws.bootstrap.WSLauncher"> 4. 5. Restart the webserver. Before logging on, submit the following URL to reset the existing log: http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1>. A screen of messages appears on the browser window. These messages are the log that has been collected. 6. 7. 8. Point to the logon URL and log on as usual. After logging on, point to the previous URL again (http://<hostname:port number>/psp/ps/?cmd=resetlog&pwd=<password from step 1>) to retrieve the log. Copy and paste the log from the browser to a text file.
24
Using Resource Analyzer

The Resource Analyzer is a new tool for the Advanced Single Server Edition of WebSphere Application Server. For information on using the Resource Analyzer, see the Version 4.0 WebSphere Application Server InfoCenter for the Advanced Edition at http://www.ibm.com/software/webservers/appserv/infocenter.html. To use the Resource Analyzer with the Advanced Single Server Edition of WebSphere: 1. 2. 3. Install the PmiSingleServerBean.ear file (available under the %WAS_HOME%\installableApps directory). Go to the %WAS_HOME%\bin directory. Enter this command: seappinstall-install ..\installableApps\PmiSingleServerBean.ear -ejbdeploy false 4. Click Enter when prompted for the Default data source JNDI name and the JNDI name. Clicking Enter enable you to use of default values as required by the Resource Analyzer. 5. 6. 7. 8. Restart the application server. Start the Resource Analyzer. Go to the %WAS_HOME%\bin directory. Enter this command: ra hostname 900 AES.
25
9/10/2010
Chapter 4 - Web Browser Configuration

Take advantage of the browsers caching ability to reduce unnecessary checking and downloading of the same Internet objects within the online transaction session. Here are the Internet objects that most likely remain unchanged during an online session: Graphics objects Style Sheets Java Scripts Some HTML pages (such as the PeopleSoft Internet Architecture login page)
The following web browser settings are the recommended configurations. These configurations are usually the default settings of the respective browsers. Verify these configurations to ensure correctness.
Microsoft Internet Explorer

Here is the configuration recipe: Internet Option->Temporary Internet Files-> Settings, Automatically
Netscape Browser
Here is the configuration recipe: Edit->Preferences->Advanced->Cache->Once per session
HTTP 1.1 Compliant Web Browser

Only HTTP 1.1-compliant web browsers request compressed files. Web browsers that are not HTTP 1.1 compliant request and receive the files uncompressed; therefore, the Web Server cannot send any compressed HTML files to the browser (despite that the configuration.properties file has the Compress Response switch turned on.) Most new browsers since 1998 and 1999 have been equipped to support the HTTP 1.1 standard known as "content-encoding." Essentially, this means that the browser indicates to the server that it can accept content encoding, and if the server is capable, it will then compress the data and transmit it. The browser decompresses the data and then renders the page. This performance feature was added for the web browsers that are using the slower bandwidth medium, such as dialup connections or 56K lines. Here is the configuration recipe to verify if Internet Explorer is configured to use the HTTP 1.1 protocol: 1. 2. 3. If using IE 6 or higher, select Tools menu -> Internet Options. Select the Advanced tab. Under HTTP 1.1 settings, verify that Use HTTP 1.1 is selected.
Currently the following browsers are HTTP 1.1 compliant: Internet Explorer version 6 onwoards
26
PeopleTools Performance Guidelines White Paper 9/10/2010 Firefox 1.5 onwards Mozilla 1.7 Netscape 7.2, 8.1 Safari 2.0.4
Confirm That the Required Browser Version Is Installed

Confirm that the required browser version to be installed is specific to your PeopleTools release and operating system from the certifications section of My Oracle Support. Browser comparison Through informal testing using PeopleTools 8.50, we have found that IE8 is faster than IE7. IE8 processes Ajax requests faster and can download images and JavaScript files much faster than IE7. Firefox 3.4 and Safari are also faster than IE7.
27
9/10/2010
Chapter 5 - Additional Configurations BROWSER COMPRESSION

From PeopleTools 8.44 on, you can enable compression between web server and browser by selecting the Compress Response check box via PeopleSoft Internet Architecture. Go to PeopleTools->Web Profile->Web Profile Configuration, select the web profile that you are using, and clear the Compress Response check box to turn off compression. Gzip and Compress are supported. The check box is selected by default. If the Compress Response References check box is selected, compression of additional resource files from web server to the browser is enabled. Only files that have their mime types specified in the Compress Mime Types field are compressed. Gzip and Compress are supported. The check box is cleared by default because there were some versions of Internet Explorer with certain settings that didn't like javascript and css in compressed format. However, that is a rare scenario; therefore, in general, this check box should be selected. The Compress Mime Types text field specifies a comma-delimited list of the MIME-type objects that should be sent in compressed form to the browser. Note that this is applicable only when Compress Response References is set to true. Gzip and Compress are supported. Here is the default: application/x-javascript,text/javascript,text/css,text/html. Note: The main purpose of compression is to reduce the amount of data to be transmitted. However, compression comes at a price: extra CPU processing time. In cases where high-speed links are used and the gain in transmission time does not justify the loss in CPU processing time, turn off compression. Note: For PeopleTools 8.41 and earlier, the compression settings would cause an error for certain updates of Internet Explorer 6.0. In particular, the PeopleSoft Internet Architecture menu may disappear when a user opens a PeopleSoft Internet Architecture page. Therefore, you should turn off these settings. This problem does not occur in PeopleTools 8.42 and later. Note: Some query entries are truncated when IE tries to open queries in Mircosoft Excel. A workaround (for PeopleTools 8.43 and later) is to turn off query compression by clearing the Compress Query check box, which turns off JUST queries.
CACHING
Caching improves system performance by reducing service requests from the web server to the application server. For PeopleTools 8.44 and later, cache settings are configured via PeopleSoft Internet Architecture. Go to PeopleTools->Web Profile->Web Profile Configuration, select the Web Profile that you are using, and then select the Caching tab. If Cache Portal Objects is selected, the portal servlet (psp) will cache the following objects in web server memory: Portal Registry Node (Remote and Local) Content Reference Template (Static only)
If Cache Portal Objects is selected, object changes won't take effect until the objects become stale and are refreshed (see Cache Stale Interval setting below) or you restart the web server. The check box is selected by default
28
PeopleTools Performance Guidelines White Paper 9/10/2010 The Cache Stale Interval is the amount of time, in seconds, before portal cache is considered stale and updated with the latest copy from the application server. In other words, this is the amount of time before changes to cached objects take effect. This setting applies to the same objects as Cache Portal Objects. Here is the default setting: 86200 (24 hours). The portal automatically throws away all cache entries in memory after the number of requests specified in Cache Purge All Hit Count. This setting applies for all websites on this web server. Setting this value to -1 disables hitcount purging. The default setting is 1000. The portal caches proxied javascripts to improve performance if the Cache Proxied JavaScripts check box is selected. The check box is selected by default. Target content is cached in memory when the TargetContent tag in the template specifies that the target should be cached. Only static content should appear in a template with a cached target tag. The TargetContent tag should look like this: <TargetContent Name="TransactionContent"><Cache Scope="application" Interval="1200" >dummy</Cache></TargetContent> You can cache pagelets using the same mechanism. The Cache Target Content check box should be selected to allow caching of target content. Clearing this check box disables all target content caching in the portal servlet, even if the target tag specifies cached content. This check box is selected by default. Homepages can also be cached on each user's browser. This means that the browser does not access the web server after the homepage is initially retrieved. You can turn this feature on or off, and also adjust the specific time interval that must pass before the web server is accessed again to get a "fresh" homepage. In any case, if a user clicks the browser's Refresh button, the homepage is accessed from the web server again, overwriting the homepage cached on the browser. Caching the homepage is beneficial in either a production or development environment. We recommend that you turn on homepage cache. The following table lists the default values of the parameters in PeopleSoft Internet Architecture: Cache-Related Properties and Default Values Cache Homepage=selected Homepage Stale Interval=1200
Homepage Stale Interval is measured in seconds. Note: The Browsers section specifies which web browser that you can use with homepage caching. Verify that your choice of browser is enabled for caching.
NAVIGATION PAGES CACHING

As with homepages, navigation pages can be cached on each user's browser. Caching on each browser improves user response time in traversing between cached menu pages. The Portal Administrator can set system-wide options for navigation cache by using the Define Personalizations page and modifying the value for METAXP. We recommend that you set the METAXP to a high value. Doing so will keep the menu pages inside the browser cache. If your menu pages are not going to change frequently, set METAXP to 10080 (one week) or more.
29
9/10/2010
Go to PeopleTools-> Personalization->.Personalization Options. Enter PPTL (PeopleTools) as the Option Category level value. Select the Format tab and then select Set Option Default Value in the METAXP row. Key in the appropriate value and click OK. Then click save when being brought back to the previous screen. Note that a change to METAXP is picked up by the application server immediately; however, because the users browsers already have cache control set by the previous value of METAXP, you must delete browser cache for the new METAXP to take effect. Users can override the METAXP value on their individual browser sessions. After logging on, go to My Personalizations, then click the Personalize Option button for General Options. Change the METAXP value by entering a new value for Time page held in cache and then click OK. Do not use navigation caching during development time because any newly added menus will not be reflected in the cache. Use navigation cache only in production systems.
HTTP KEEPALIVE
HTTP KeepAlive is intended to maintain a persistent socket connection between the web browser and the web server so that no new connections are required when the HTML pages refer to other HTML objects. However, keeping the socket connection persistent occupies a socket pair between the browser and web server. When the KeepAlive timing is set too long, the following problems occur: The web server must manage many idle connections. Then number of new connections available is reduced.
We generally advise that those with PeopleTools 8.4x turn on KeepAlive. KeepAlive is the default for Oracle Application Server, WebLogic, and WebSphere.
REDUCING TCP WAIT TIME

The default TCP Wait Time for both NT and most UNIX operating systems is 4 minutes, which is too long for PeopleSoft Internet Architecture usage and tends to leave too many socket connections staying in the TIME_WAIT state. By shortening the TCP Wait Time, the socket can be recycled more efficiently.
For Windows NT
Use the registry editor, regedit, and create a REG_DWORD named TcpTimedWaitDelay under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TcpIp\Parameters. Set the value to 60 secs (seconds).
For AIX
To see the current TCP_TIMEWAIT value, run the following command: /usr/sbin/no a | grep tcp_timewait To set the TCP_TIMEWAIT values to 15 seconds, run the following command: /usr/sbin/no o tcp_timewait =1 The tcp_timewait option is used to configure how long connections are kept in the timewait state. It is given in 15-second intervals, and the default is 1. Use the default or change the value to 2 for slower networks.
30
PeopleTools Performance Guidelines White Paper 9/10/2010 Note: Be careful when you use no command, which performs no range checks and instead accepts all values for the variables. If used incorrectly, the no command can cause your system to become inoperable. The no command operates only on the currently running kernel. The command must be run again after each startup or after the network has been configured.
For HP/UX 11 or Solaris 2.8 and Later

As root user, execute the following command: ndd -set /dev/tcp tcp_time_wait_interval 60000 where 60000 is for 60 secs, the minimum recommended value. You must put this command in one of the rc2.d scripts to get it to set automatically at boot time. For HP/UX 11, you can place the ndd settings inside the /etc/rc.config.d/nddconf file without the need of the startup scripts.
For Solaris Prior to 2.8

Use the parameter name tcp_close_wait_interval instead of tcp_time_wait_interval.
AIX THREAD MODEL

IBM has recommended the following environment variables setup to improve the Threading model with PeopleSoft Internet Architecture. Set up the following environment variables for the PeopleSoft Application Server user, database user, and web server user: For Korn Shell user, place the following lines in the .profile: export export export export export export AIXTHREAD_SCOPE=S AIXTHREAD_MNRATIO=1:1 AIXTHREAD_COND_DEBUG=OFF AIXTHREAD_GUARDPAGES=4 AIXTHREAD_MUTEX_DEBUG=OFF AIXTHREAD_RWLOCK_DEBUG=OFF
For C Shell user, place the following lines in the .cshrc file: set set set set set set AIXTHREAD_SCOPE = S AIXTHREAD_MNRATIO = 1:1 AIXTHREAD_COND_DEBUG = OFF AIXTHREAD_GUARDPAGES = 4 AIXTHREAD_MUTEX_DEBUG = OFF AIXTHREAD_RWLOCK_DEBUG = OFF
TIMEOUT SETTINGS
There are in general three types of timeouts:
31
PeopleTools Performance Guidelines White Paper 1. 2. 3.
9/10/2010
Timeout during a PeopleSoft Internet Architecture transaction (intratransactional): When the timeout expires, the transaction fails and must be resubmitted. Timeout between PeopleSoft Internet Architecture transaction (intertransactional): When the timeout expires, the user has been idling for too long, and the resources associated with the user are freed. You must relogin to re-establish the state. Timeout that "reserves" a resource until the expiration time before putting it back into the pool for recycling. (for example, tcp_timewait).
Note that for the intratransactional timeouts, their values should be staged. In other words, the end-to-end timeout value should always be greater than that of its intermediate legs. With this in mind, lets review our timeout values, from bottom up: Appserver: Service Timeout = x sec (default 300) (in psappsrv.cfg) Type=intratransaction, which includes the time it takes at the DB server. According to the PIA Answer Book:
Service Timeout Specifies the number of seconds a PSAPPSRV waits for a service request, such as MgrGetObj or PprLoad to complete, before timing out. Service Timeouts are recorded in the TUXLOG and APPSRV.LOG. In the event of a timeout, PSSAPSRV terminates itself and Tuxedo automatically restarts this process. In other words, it has to be large enough to accommodate the longest acceptable requests and queries. It is recommended to set x=1200 (20 minutes).
PeopleSoft Internet Architecture: tuxedo_send_timeout = w sec (default 50) ; tuexdo_receive_timeout = x sec (default 600) (in pstools.properties) The receive timeout is also intratransactional, and it has to be bigger than appserver service timeout. The receive timeout indicates the maximum number of seconds that the servlet waits for a response from the application server. If you increase your application server service timeouts, such as the Service Timeout setting for PSAPPSRV, then increase the tuxedo_receive_timeout parameter to be greater than the Service Timeout values that appear in the PSAPPSRV.CFG configuration file on the application server.
Servlet: sessionTimeout = x sec (default 1200) (in configuration.properties) This is intertransactional, as one has to relogin when the servlet expires (technically this is a security setting). The meta refresh tag is in seconds and should be less than or equal to the session.timeout for the servlet. For WebLogic 6.1, the meta refresh tag should be less than or equal to <session-timeout> as discussed in the following section.
JOLT: Client Cleanup Timeout = x min (default 60) (in psappsrv.cfg) This is intertransactional. See description (default is 60, but in most cases this can be reduced to 10 to conserve resources): Client Cleanup Timeout: Specifies the amount of time, in minutes, that a client connection can remain idle (no work requested) before Tuxedo terminates a client connection. Client disconnects are transparent to a client. To reconnect, users must only click the mouse.
Webserver session timeout: <session-config><session-timeout>x</session-timeout></session-config> (in <webserver home dir>/<peoplesoft web domain>/PORTAL/WEB-INF/web.xml)
32
PeopleTools Performance Guidelines White Paper 9/10/2010 This is intertransactional and specifies the number of minutes (WebLogic) to wait before invalidating an unused session. Note that setting this value too high ties up web server resources, especially when users close their browsers instead of logging out. Setting it to be the same as (or a bit higher than) the JOLT cleanup timeout is generally a good idea. HTTP timeout: Apache: Timeout = x sec (in httpd.conf) HTTP timeout, unfortunately, serves as both intertransactional and intratransactional in different scenarios, so it may or may not be higher than the rest of the timeouts. This directive defines the amount of time Apache will wait on three occasions: 1. 2. 3. The total amount of time it takes to receive an HTTP GET request. The amount of time between receipt of TCP packets on a POST or PUT request. The amount of time between acknowledgements on transmissions of TCP packets in responses.
A common problem: Sometimes in PeopleSoft Internet Architecture, time out error occurs on PeopleSoft pages even when people are using the page. Solution: 1. Increase the timeout values in servlet session timeout and webserver session timeout. In configuration.properties: sessionTimeout = 3600 sec In web.xml: <session-config> <session-timeout>60</session-timeout> </session-config> 2. Increase values in configuration.properties in Apache Group->Apache\htdocs\PeopleSoft. Set "meta-tag" session timeout to 3600 (which enables users to use the page for 60 minutes with no time out errors). 3. Increase the values in pstools.properties if long queries are common: Set tuxedo_receive_timeout to 1500.
33
9/10/2010
Chapter 6 - Integration Broker CONFIGURE PSBRKHND MESSAGE BROKER HANDLER

The primary purpose of this handler is to determine the appropriate routings based on the Tuxedo request received and update the appropriate database tables (queues) accordingly. This handler also executes the OnRoute PeopleCode events (OnRouteSend or OnRouteReceive) and Inbound Transformations. Note that the processing time to determine the routings is much faster then actually processing a PeopleCode event. The number of OnRoute and Transform events is typically low compared to the overall number of different messages. If physical resources are a problem, you can reduce the number of this handler without impacting performance.
OPTIMIZE MESSAGE QUEUE VERSUS MESSAGE CONSUMPTION

There is a difference between the messaging system being able to receive a given number of messages per slice of time and being able to process the message data (that is, a notification process) per slice of time. For example, if a non-PeopleSoft system needs to send messages at a rate of x number per minute, the PeopleSoft messaging system can receive and queue them at this rate. But if the notification process involves a lot of business logic or actions that take up a lot of server resources, the application may not be able to consume the messages at the same rate that the messages are queued by the messaging system (given a particular hardware configuration). You should determine what the real requirement is for slice of time throughput. Is the requirement the ability to have the messages queued at x number per minute (with the data from x messages being processed in more than that minute), or does the message data from x messages absolutely have to be run through the business logic within that minute slice of time? If the messages must be queued (that is, must you receive them as fast as other systems are sending them), then you should explore additional configuration options for optimization. You might even look at deferring message consumption (running the business logic) until certain times of the day or night when additional system resources may be available. If the data from x messages absolutely has to be run through the business logic in that one minute slice of time, then you may have to look at what is happening in the notification event and optimize that.
USE OF MESSAGE SEGMENTS FOR ORACLE DELIVERED FULL SYNC SERVICE OPERATIONS
Messages (service operations) should be designed to take full advantage of as much queue partitioning as possible. One area of concern is the Oracle-delivered full sync service operations. These particular service operations do not take advantage of partitioning due to the way that they currently have to chunk the content data and process the notifications. Integration Broker has come up with an alternative to this current chunking mechanism using message segments. Message segments provide a way to process and send large amounts of data (Gbytes) without impacting performance due to PeopleCode processing, or running out of memory. Message segments enable a single message to load with all the data segmented by any chunkable size desired. After one segment is populated based on a configuration parm, or overridden by PeopleCode, that segment is serialized to XML and inserted into the IB database queue compressed. The next segment is then available for processing. This type of loading continues until the message is completely loaded with all the desired data. The message can be sent either as one message (ordered segments) or multiple messages (unordered segments). The actual data is sent chunked by segments to the gateway and received by the target system. These segments should be complete stand-alone data structures. The message object is responsible for this memory management. The consumption of a segmented message in PeopleCode is straightforward. One segment at a time is
34
PeopleTools Performance Guidelines White Paper 9/10/2010 decompressed and serialized into a rowset. After the data is inserted into the database, the rowset is destroyed, freeing up memory, and the next segment is then loaded. This process repeats for the number of segments in the message. Refer to IB PeopleBooks for Message PeopleCode APIs with respect to message segments.
OPTIMIZE ONE MESSAGE VERSUS MULTIPLE MESSAGES PER QUEUE

Another design-time consideration when creating queues is whether to have one message per queue or many messages in one queue. The answer to this question depends on many factors. For example, if there are 20 unrelated messages in a queue, the dispatcher will try to process the 20 messages in the queue assuming these messages are partitioned. If you create a queue for each message, there would be 20 queues. The dispatcher would have to traverse all 20 queues to process those 20 messages, which would compromise performance due to more dispatch cycles and database reads and writes. Therefore, ideally you should put related messages in one queue and partition accordingly. For high volume transaction messages, create a queue for each message, as these queues can then be part of a dedicated messaging server.
TUNING PSADMIN PARAMETERS FOR ASYNCHRONOUS MESSAGING

Many configurable parameters in PSADMIN have significant impact on performance of asynchronous-based messaging. This section discusses each of these parameters in detail and its impact on performance. Tuxedo Queue Size: The actual Tuxedo message queue size. For NT, the Tuxedo queue size is a registry parameter. For Unix systems, it is a kernel parameter. This value is used for Tuxedo queue threshold determination by the pub/sub dispatchers. This value should be 1 or 2 MB instead of using the default of 64 KB. Note that for AIX, the Tuxedo queue size is dynamic; therefore, you should use an arbitrary value. Otherwise, the benefits of throttling are not realized. As Tuxedo requests get queued, there is a point where the queue can get full and any additional requests get disregarded. From 8.43, the dispatcher reads the number of queued requests and determines, based on this parameter, if any or all requests should be sent out for this cycle. Therefore its important that this value accurately reflects the actual size.
DISPATCHER Parameters
Recycle Count: The dispatcher automatically recycles itself when this value is reached. By default, the recycle count is set to 0. This count is based on the number of Tuxedo requests. In general, you should not have to cycle the dispatcher. However, if a recycle count is used, it will affect performance because initialization will be performed which includes rebuilding all the inmemory queues for each active queue assigned to that dispatcher. Dispatch List Multiplier: This is a parameter used to throttle the number of requests sent to its associated handlers. The actual list is the number of associated handlers multiplied by this value. The current default is set to 10. This value was obtained by many performance tests. You should not have to change this value because it scales well. Scan Interval: This is the interval that the dispatcher runs its on-idle processing. The current value is set to 15, which is fine if Pub/Sub is running in the same domain as the PeopleSoft Internet Architecture appservers. However, if the Pub/Sub servers are stand-alone, this value should be set to 1, as this is the only mechanism to initially poll the database queue for work. Dispatcher Queue Max Queue Size: This value is the maximum number of items (messages) per queue that the dispatcher retains in memory. The current default is set to 1000, which was obtained after many performance tests. You should not have to change this value because it scales well. Memory Queue Refresh Rate: This is the number of actual dispatches that the dispatcher will automatically rebuild in its inmemory queue for a particular queue. The queues should not get corrupted; however, the current default value of 1000 is set at such a high level that it does not impact performance and is recommended based on performance tests. You should not have to change this value because it scales well.
35
9/10/2010
Restart Period: This is the time that the dispatcher attempts to redispatch messages that are still in the START status. This value can potentially have a big impact on overall performance of the messaging system. When the dispatcher dispatches a request, the system sets the status of the message to START. The Tuxedo request is queued, and the next available handler will attempt to process this request and set the status to WORK. However, when the message system is under configured (that is, there are not enough handlers to process all the requests), the request stays queued. The dispatcher again sends the request after the restart period has elapsed. This potentially leads to a lot of redundant requests that the available handlers must cycle through. This leads to the Tuxedo queue overflowing and potentially losing requests, request that must be picked up when the restart period is reached. However, you do not want to set this value too high, as messages would not be restarted in case of a handler crash. A good guideline is to use the number of incoming requests per second divided by the number of associated handlers, multiplied by the average processing time per second. Use this formula: ((Incoming requests per second)/ (# of associated handlers)) * (average processing time per request)
PSPUBDSP only
Ping Rate: This parameteralong with the scan intervaldetermines how often a node that is in the PSNODESDOWN table should be pinged to see if it is in a valid state to send a request to. Part of the on-idle processing performs these ping requests. When there are a lot of nodes that are down due to improper configuration of routings on Service Operations, many CPU cycles are spent performing these pings. This value allows for a longer time between subsequent pings. Here is the algorithm used to determine the interval: Attempts * Ping Rate * Scan Interval) Maximum Ping Interval: This value is the maximum time between subsequent pings for a node that is in the PSNODESDOWN table. This value is in hours.
HANDLER Parameters
Min Instances and Max Instances: These values should always be the same. If the Min Instance is not the same as the Max Instance, then under load, the system has to wait for the process (the handler) to boot up and initialize before it can be used. It is better to boot all at one time, avoiding allocations during max load. Recycle Count: Because the handlers run IB Events that contain PeopleCode, memory leaks are likely. Therefore, you should have a high recycle count and monitor it to determine if it grows to an undesirable size. The recycle count by default is 20000; however, depending on the PeopleCode being run, you can set this value higher or lower. The problem with a recycle count on a handler is that with proper load balancing from Tuxedo, all associated handlers recycle at approximately the same time. If this becomes a problem, set the recycle count to 0 and create an automated script that will stagger the recycling of the handlers by using Tuxedo command line arguments to specify the specific handler. Max Retries: This value represents the number of times that the handler will attempt to process a message before the status is set to TIMEOUT. Therefore, if the PeopleCode being run causes a crash of the process, it will attempt to process it again. This value should be set to a low value5 or lowerto limit the amount of handler crashes for one specific bad message.
PSPUBHND only
Thread Pool Size: This value represents the number of concurrent threads that the handler can spawn for HTTP requests. Note that with performance benchmark testing in this area, with a thread count set to 5, one handler was able to replace three nonthreaded handlers. This is something to consider if memory resources are scarce.
TUNING PSADMIN PARAMETERS FOR SYNCHRONOUS MESSAGING

36
PeopleTools Performance Guidelines White Paper 9/10/2010 These are the configurable PSADMIN parameters for synchronous messaging that can affect performance. This section discusses each of these parameters in detail and its impact on performance. Min Message Size For Compression: Size of message content data that will cause the Integration Broker to compress the data prior to posting to the Integration Gateway. In general, Sync messages should be as small as possible to improve response time. Third-parties should always use compression if possible. Thread Pool Size: This value represents the number of concurrent threads that one PSAPPSRV process can spawn for HTTP requests. The default is set to 5 based on performance benchmarks.
CONFIGURE DEDICATED MESSAGE SERVERS FOR HIGH VOLUME ASYNCHRONOUS MESSAGING

It is best to run pub/sub (asynchronous) messaging activity on dedicated servers when processing a high message volume. These domains should have application server processes (PSAPPSRV) configured in addition to pub/sub servers. The advantage of having the application server processes on the pub/sub machine is that if the server has only pub and sub services defined, it must poll the database queue to determine if there is work to be done. With the (additional) application servers configured on the same machine as the pub and sub services, you get event-driven notification of work to be done without the latency of polling. The gateway should point to this application server domain for message processing. PeopleSoft Internet Architecture should not be accessed on the dedicated machine. To access PeopleSoft Internet Architecture for configuring messaging, use the PeopleSoft Internet Architecture install on the other online application servers. Dedicated messaging servers are assigned a certain number of queue names that result in its queue list. This list must be unique across messaging servers pointing at the same database. The default messaging server defined by _dflt at the end of each dispatcher process creates its own queue list based on all queues in the database, less the queues defined for the same type dedicated server process. Therefore, to configure multiple dedicated servers on different domains (to spread across different machines), ensure that the queue lists are indeed unique for the different dispatcher types; otherwise, there will be database contention and the overall performance will be significantly impacted. In addition, add only the server type needed for the queue. This means that if you have long running notifications, create only a subscription contract server. Moreover, remove all queues no longer in use, as these will consume additional resources. For information on how to configure dedicated servers, reference the Integration Broker PeopleBooks.
CONFIGURE DEDICATED MULTIPLE DOMAINS FOR HIGH-VOLUME ASYNCHRONOUS MESSAGING

Pub/sub domains for high-volume messaging should usually run on a box other than that used for online users. Doing so helps prevent messaging traffic from being affected by the number of online users, and vice versa. Separate pub/sub domains work only for Asynch messaging. Synch messaging runs as a service in the online app server. When running pub/dub on a dedicated box, adjust the Scan Interval for each dispatcher in PSADMIN to 1, as this will be the fastest initial poll rate from the database queue. Within a dedicated pub/dub machine, you can configure one or more domains. There are few scenarios in which having multiple domains on one box will result in performance advantages. The reason is that each domain has a publish dispatcher and a subscription dispatcher. The roll of the subscription dispatcher is to take messages from the database queue and create Tuxedo calls to get them processed. The dispatcher is where the partitioning logic is applied. The dispatcher looks at the queue, figures out how many messages can be processed next (based on partitioning), and then puts them in the Tuxedo in-memory queue to be processed. The dispatcher locks the rows in the DB message queue until the Tuxedo calls come back with either success or failure. The dispatcher doesnt wait for the return of the first set of calls before moving on. It will continue to read the DB queue and make the Tuxedo calls even before the first set of calls have come back. The second dispatcher may not be able to read from the DB queue because the first dispatcher locks certain rows while processing. The second dispatcher can also
37
9/10/2010
add complications in debugging. Adding another domain takes up systems resources. If you are going to consume resources, then add handlers rather than a new domain. If you are trying to configure for failover, configure using IB failover.
APPLICATION GUIDELINES FOR ASYNCH MESSAGING

You should partition messages as much as possible. Third-party clients should post to PeopleSoft in parallel. To reduce delays due to posting/queuing, use large messages (multi-transaction) for full sync processes, and use compression. When publishing from PeopleCode, use the enqueue option as much as possible, or publish the message as late in the transaction as possible. If you publish a message early in a transaction, and the transaction takes a long time to complete, a lock will be maintained on the queue sequence ID table, blocking other messages in the same queue. Notification PeopleCode should be as light as possible. For multitransaction messages, commits should be frequent to minimize contention. CIs should not be used for full sync processes. They may be used in incremental sync processes where volumes are not expected to be high. Note that CI performance is dependent on the underlying component. If the component is light, it is possible to have a fast CI. The heavier the base component, the slower the CI. GetMessage is a relatively heavy call. You should use and reference the message thats passed into the method whenever possible in PeopleCode. Transform code (either PeopleCode or XSLT) should be as light as possible. In general, XSLT transforms will be much faster than PeopleCode transforms. Message compression is automatic for PeopleSoft-to-PeopleSoft transactions. For third-party applications, set the HTTP target connector property sendUncompressed to N. Compression can reduce response time by 20 to 80 percent based on message size. By adding a simple relay servlet, third-party messaging can take advantage of compression. Areas to review when using the Full Sync EIPs: 1. Database Tuning: Create database statistics in a test environment from test runs and export them before production runs. Full Sync EIPs are used for initial database creation. Empty databases do not have database statistics. If EIPs are using temp tables, make sure the indexes are available for reads. After temp table population, create the database statistics for these tables. Archiving: Enable the archive option on the Queue page for queues designated for EIPs. This will cause the message data to be archived into other IB PeopleTool tables. The Full Sync EIPs are one-time in nature and hence do not need to be archived. Database Layout: Plan database layout by splitting PeopleTool tables from application tables for better throughput. Additional Tips: Enable archiving for Full Sync EIPs or any process where archiving is not required. Deactivate all message queues that are not being used. After Full Sync is run, the DBA should update the database statistics for the new (grossly changed) table sizes. Keep commits at the transaction level and not at the message level. This will reduce locking and improve performance.
2.
3. 4.
Application Guidelines for Sync Messaging

Response time tends to be important in synchronous requests, so transforms and OnRequest PeopleCode should to be light. Use CIs only if the base components are light and have quick response times. Keep the number of sync requests from a client to a minimum or use synchronous threading. For example, if you have a certain amount of work to be done on a remote system, pack as much of the work into as few of calls as possible. Minimizing calls reduces the amount of PeopleCode message overhead to instantiate request/response messages. In general, sync messages should be small to improve response time. Third-party clients should use compression, if possible (we do this by default between PSFT applications). Partitioning is not an issue with sync messaging. Sync messaging should
38
PeopleTools Performance Guidelines White Paper 9/10/2010 typically be used for remote queries, not for remote inserts/updates/deletes. If synch messaging is used for inserts/updates/deletes, SyncRequests do not share a transaction context with the requesting system. If a SyncRequest has been completed, and the client transaction rolls back, the SyncRequest will not be rolled back. (Publishes will be rolled back). Also, a component should not depend on the successful completion of a SyncRequest. If the remote system is down, the SyncRequest will fail. The requesting application should be prepared to deal with this by using exception logic, or should use the Asynch system. If the remote system is down, the Asynch system will retry. If you need to send data to multiple systems at the same time, use threaded sync requests or possibly Asynch messaging. The Asynch system is much more efficient at fanning out information to multiple target systems.
CONFIGURE LOAD BALANCE INTERVAL (PEOPLETOOLS 8.48 AND LATER)

During on-idle processing, the Integration Broker dispatcher will ping nodes that are down (Publication Dispatcher only) and also synchronize its in-memory queue with the database queue table. If the dispatcher is constantly busy, the on-idle processing may not get run for extended periods of time, resulting in failed messages not getting retried or new messages not getting processed. In PeopleTools 8.48 and later, a new configuration parameter in psappsrv.cfg is called Load Balance Interval. Setting this parameter (measured in minutes) forces the dispatcher to perform on-idle processing at fixed intervals, which could help avoid queues not getting processed due to the previously mentioned reasons.
MASTER/SLAVE
When setting up a master/slave configuration using machines with different CPU processing power, use the more powerful machine as the master. If you use the slower machine as master, that machine may not keep up with the slave. As a result, the slaves would sit idle and throughput would not be maximized.
REDUCING MAXIMUM APP MESSAGE SIZE

Some operations may benefit from smaller message size. For example, when running FULLSYNC message from a HCM instance to setup Enterprise Portal Resource Finder, reducing the maximum app message size from default of 10MB to 25K enables the process to run significantly faster. To reduce the maximum app message size, go to PeopleTools->Utilities>Administration Options-> PeopleTools Options and change the Maximum App Message Size field. We suggest manually setting the value back to the default after the FULLSYNC messages are complete because this setting affects all message types.
MULTITHREADED INTEGRATION BROKER (PEOPLETOOLS 8.46 AND LATER)

Synchronous Messaging: PeopleTools 8.46 provides a multithreaded version of IntBroker.SyncRequest() and enables users to configure the number of message handling threads. In general, setting the thread count equal to the number of psappsrv.exe processes in the target domain for the messages can improve message handling response times by as much as 80 percent over a single threaded implementation. Note: Make sure that you apply PeopleTools 8.46.12 or later and PeopleTools 8.47.06 or later if you are not on PeopleTools 8.48. These patches fix an important thread leak issue associated with using this feature. Asynchronous Messaging: As with synchronous messaging, we recommend setting the number of threads for PSPUBHND equal to the number of psappsrv processes in the target domain. Note: If needed, you can distribute PSPUBHND via a Master/Slave configuration if the number of threads is limited by the source domain hardware.
39
9/10/2010
SCAN TIME SETTING FOR APP MESSAGING

An Oracle database user who has database logging turned on may see a large volume of archive log created. The contents of the Oracle Archive log contained the following: SET TRANSACTION READ WRITE; INTERNAL; COMMIT; The Archive Log (or Redo log) is a side effect created by the Application Messaging Dispatchers select statement, which is invoked periodically based on the value of the Scan Interval. Application Messaging has been enhanced to minimize the side effect on Oracle logging.
PUBSUB ERROR AND APP SERVER LOG FILE GROWING

Problem: These PSSUBDSP_dflt errors were occurring upon starting the App server. There are hundreds of entries like this. While these errors occurred, no access to the PS system was possible in three-tier mode. Two-tier mode invoked no errors. PSSUBDSP_dflt.1784 [10/24/00 13:46:22](1) (NET.333): The service PSSUBHND_dflt could not be sent because of a blocking condition within Tuxedo on the client. PSSUBDSP_dflt.1784 [10/24/00 13:46:22](1) (NET.333): The service PSSUBHND_dflt could not be sent because of a blocking condition within Tuxedo on the client. PSSUBDSP_dflt.1784 [10/24/00 13:46:22](1) (NET.333): The service PSSUBHND_dflt could not be sent because of a blocking condition within Tuxedo on the client. Second Error: >PSSUBHND_dflt.2253 [10/24/00 13:46:25 SubConProcess](1) (NET.346): Failed to execute SubConProcess request PSSUBHND_dflt.2253 [10/24/00 13:46:25](2) Service SubConProcess failed Resolution: This problem was appearing when the nightly backup inadvertently shut down the database while the appserver was running. When we tested shutting down everything in the correct order, the problem disappeared. The PSSUBHND instances were too busy erring out to handle even one subscription. As long as you follow the correct shut down procedures, you can avoid this problem. If, however, this problem does occur, shut down the web server, application server, and process scheduler (not the DB) and restart them in the correct order.
40
Chapter 7 - Operating System Settings

For a large environment with many domains, use the following kernel parameters. Kernel parameters of this size will provide a good deal of resources and are not uncommon in our customer environments.
FOR LINUX
The default settings and available tuning options vary from one Linux kernel version to the next, so information presented in this section may be incomplete. The correct settings also vary depending on factors such as available physical memory, number of processors, and PeopleTools component configuration. <h4 These settings can be found in the /etc/sysctl.conf file, which is loaded when the system boots. # raise max open fd limits fs.file-max = 65536 # use wider range for local ports, # setup all Tools components to use port numbers between 1024 and 10000 net.ipv4.ip_local_port_range = 10000 65000 # allow more/larger IPC message queues kernel.msgmni = 512 kernel.msgmax = 1048576 kernel.msgmnb = 1048576 # set IPC shared memory blocks, # these are system defaults, # except for shmall which is varies depending on physical memory kernel.shmmni = 4096 kernel.shmmax = 33554432 kernel.shmall = 1073741824 # adjust IPC semaphore settings kernel.sem = 250 256000 64 1024 After you change the sysctl.conf file, you can load the updated values from that file into your running system with sysctl -p.
TCP_WAIT
Unlike other systems, TCP_WAIT is not an adjustable parameter. For reliable TCP behavior, do not lower TCP_WAIT below 60 seconds on any system. In the Linux kernel, the parameter is hard coded at 60 seconds and requires a recompile to change. To prevent DOS attacks, the number of sockets that can be in a TCP_WAIT state is limited. The sysctl name is net.ipv4.tcp_max_tw_buckets and the default value is 180000. If you use this default, you must establish, process, and close more than 3000 sockets per second to see a problem.
FOR SOLARIS
MSGMAP=2048
41
PeopleTools Performance Guidelines White Paper MSGMAX=65536 MSGMNB=65536 MSGMNI=1024 MSGSEG=32767 MSGTQL=1024 SEMMAP=512 SEMMNI=512 SEMMNU=4096 SEMUME=10 SEMMNS=4096 SEMMSL=8
9/10/2010
FOR HP-UX
dbc_max_pct=10 dbc_min_pct=8 default_disk_ir=1 fs_async=1 max_thread_proc=2048 maxfiles=2048 maxfiles_lim=4096 maxssiz=200802304 (191.5 MB) maxswapchunks=2048 maxuprc=512 maxusers=2000 msgmap=2048 msgmax=65535 msgmnb=65535 msgmni=1024 msgseg=32767 msgtql=2046 semmap=512 semmni=512 semmns=4096 semmnu=4096
FOR TRU64
ipc: msg_max msg_mnb msg_mni msg_map msg_tql shm_max shm_min shm_mni shm_seg sem_mni sem_mnu sem_mns sem_msl sem_opm sem_ume sem_vmx sem_aem = = = = = = = = = = = = = = = = = 262144 262144 16384 20000 4096 4294967295 1 300 100 4096 1048 819200 1000 400 1048 32767 16384
42
PeopleTools Performance Guidelines White Paper 9/10/2010 ssm_threshold=8388608 num_of_sems = 1024 max_kernel_ports = 22487 inet: tcpnodelack = 0 tcbhashnum = 16 tcbhashsize = 8192 ipqmaxlen = 1024 ipqmaxlen = 2048 ipqs = 1 ipqs = 32 tcp_mssdflt = 1460 tcp_msl = 60 pmtu_enabled = 0 tcp_sendspace = 61440 tcp_recvspace = 61440 udp_ttl=60 vfs: bufcache = 1 fifo_do_adaptive = 0 socket: somaxconn = 65535 sominconn = 65535 proc: per_proc_data_size = 3221225472 (3072MB, or three-quarters of the value of perproc-address-space) per_proc_address_size = 4294967296 (4096MB) This is needed if you must compile large numbers of Cobol programs. Many applications, for example Oracle and the Tru64 Java Fast VM, require that you properly set per_proc_data_size. However, the per_proc_data_size should be less than the physical memory available on the system; otherwise, swapping will occur.
43
9/10/2010
Chapter 8 - Call Telephony Interface (CTI) RENSERVER PARAMETERS (PSRENCONFIG.TXT)

reaper_interval
How often (in sec) to look for and delete expired events. Default is 3600 (1 hour), which caused a steady increase in RENSERVER memory.
ns_param default_kn_expires reaper_interval 600 (Recommended)
Default to use if an event is received without a kn_expires header. Use "infinity" or something like "+3600" for 1 hour; must be at least "+5." Default is 3600 (1 hour), which caused a steady increase in RENSERVER memory.
ns_param permission_max_idle default_kn_expires "+10" (Recommended)
How long (in sec) the permission can stay in cache if the tunnel is closed. Default is 60, which increases the time the PSTOKEN of the agent is expired at the RENSERVER.
ns_param mtu_size permission_max_idle 300 (Recommended)
New parameter to pad the network TCP packets with additional data to reduce latency in TCP ACK packets.
ns_param mtu_size 1500 (Recommended)
PSMCAPI PARAMETERS (RENCLIENT.PROPERTIES)

interval_topic_reaper
Topic reaper interval in milliseconds. Default is 30000 (30 seconds), which keeps the session alive for an extended period of time.
interval_topic_reaper = 3000000 (Recommended) heartbeats_to_miss
New parameter specifies the number of heartbeats to miss before the session is set for deletion. Default is 2.
44
PeopleTools Performance Guidelines White Paper 9/10/2010 heartbeats_to_miss = 10 mtu_size (Recommended)
New parameter to pad the network TCP packets with additional data to reduce latency in TCP ACK packets.
mtu_size=1300 (Recommended)
PSMCAPI JAVA OPTIONS (STARTSERVER.BAT)

Java Heap Size Options
Default is Xms256m Xmx512m. Increase to below recommended value to handle higher call rate (more than 10 cps).
-Xms512m -Xmx768m (Recommended)
45
9/10/2010
Chapter 9 - PSAE and PSAESRV

The benefits of PSAESRV versus PSAE are a popular topic of discussion. Our studies have shown that PSAE is as good as PSAESRV for most practical purposes. If you have an application engine job that runs longer than 10 seconds, PSAE is equivalent to PSAESRV. PSAE has the added advantage of being recycled at the end of each application engine job, cleaning up any outstanding SQL cursors to the database that may have been left behind. Because PSAE recycles after each use, PSAE does not have any possible memory leakage problem that may occupy the precious system memory. In short, PSAE is a cleaner workhorse. To shutdown PSAESRV, when you configure the Process Scheduler, you can change the default of the PSAESRV instance to 0. Values for config section - PSAESRV Max Instances =0 Recycle Count=1000 Allowed Consec Service Failures=2
46
Chapter 10 - Reporting Tools XMLPUBLISHER (PEOPLETOOLS 8.48 AND LATER)

Consider the following tips when using XMLPublisher: Increase JVM heap size to 512MB. See the "Set Application Server JVM Options" section on how to do this. Use PDF for reports where the PDF template will suffice. For more complex reports that require the RTF template, take greater care when designing the RTF template and data source. Use SQR or other mechanisms to generate the XML File as a preprocessing step. Reason: the XML File is the most optimal data source for an XMLP report. While PSQuery and rowset data are supported data sources, they will, in effect, be converted to XML File during the report processing. Using SQR or other mechanisms is one optimization technique.
BUSINESS OBJECTS ENTERPRISE (PEOPLETOOLS 8.48 AND LATER)

For customers who have a large reporting load and have converted the report format to Business Object Enterprise 11 (BOE), to avoid the reporting load affecting other online users, we recommend that you configure a dedicated application server domain and install the BOE server on a separate machine. BOE will request data via the integration gateway and Query Access Services (QAS). QAS will make requests to PSANALYTICSRV to retrieve the actual data. You should set the Max Instance Count of PSANALYTICSRV to a value equal to or greater than the maximum number of concurrent reports that you must run. By default, PSANALYTICSRV recycles after each request. If you are running many small reports that take less than 1 minute to finish, it may be beneficial to set recycle count to a greater value to minimize the cost of restarting PSANALYTICSRV. However, if you are running large reports, the memory footprint of the PSANALYTICSRV may be high, and it is better to set the recycle count to 0. In addition, if there is a burst of report requests, the PSANALYTICSRV processes spawned to handle the request will continue to consume memory if you set recycle count to greater than 0 (eventually the idle processes will shut down). Depending on the size of the document, when generating BOE reports, lots of physical files are created in various layers: BOE temporary files, QAS repository files, and Report Repository files. For better performance, configure these locations such that the physical files are created on fast disks that are optimized for faster read and write. While submitting BOE reports through your Peoplesoft application, set the Minutes Before an Idle Connection is closed setting to a lesser value (1 minute) to free up resources from inactive users more quickly. You can change this setting from the BOE Management Console.
47
9/10/2010
Chapter 11 - Monitoring Tools

PeopleSoft Ping
PeopleSoft Ping is a convenient monitoring facility using PeopleSoft Internet Architecture, and it shows the time spent in different tiers of the PeopleSoft system. Select PeopleTools->Utilities->PeopleSoft Ping. See PeopleBooks for more details.
psadmin
You can use psadmin to monitor Tuxedo queueing. You can use this information to determine the optimal number of PSAPPSRV processes. See the Application Server Guidelines section for more details.
PeopleSoft Performance Monitor

You can capture detailed performance information for individual requests using PeopleSoft Performance Monitor (PPM). PPM provides a breakdown of the response time for the request and enables you to see time consumed by individual SQL statements or PeopleCode executions. For more information on PPM, see the White paper entitled "PeopleSoft Performance Monitor" on My Oracle Support. The Document Id is 747510.1
48
Chapter 12 - Version 8.50 GUI Enhancements

Version 8.50 Enhancements
Version 8.50 includes many GUI enhancements that improve the look and feel of the online experience. Although these enhancements affect all of the tiers in your PeopleSoft environment, we have not encountered any required tuning or configuration above and beyond what was already being performed for release 8.49. The information in this chapter is intended to explain how some of the GUI enhancements actually work to give you a better understanding of how they might affect your environment.
Autocomplete (aka Type Ahead)

Autocomplete vastly improves the user experience. It is enabled for all prompt fields. It allows the user to perform partial searches on prompt fields without having to navigate to the prompt page. In one step, the user can enter search data, wait for the list to populate, and then choose from the list of retrieved values. This reduces the amount of typing and clicks that the user has to perform to find the desired value. This feature can be disabled via a user personalization change. The amount of time that the browser waits before requesting a list of values is .5 seconds. This value cannot be changed.
Swan Look and Feel

To achieve a more modern look and feel requires additional images, style sheets and Javascript. The Swan Look and Feel results in additional requests to your webserver including: More javascript More images More CSS (Cascading Style Sheets)
Once cached on the browser, there are no more requests for these files.
Partial Page Refresh (aka Ajax Updates)

To achieve Ajax updates for a page required a new infrastructure on the browser to handle the responses as well as additional Javascript processing. Javascript manipulation and XML processing can slightly increase the amount of time required by the browser to load the page. This is due to the browser HTML engine applying the new updates to the HTML page. Ajax has been shown to reduce the number of bytes transferred to the browser in some situations when partial page updates occur. This reduces your network utilization.
Modal Windows
Modal Windows are used for a number of tasks including messages prompt look ups secondary pages 49
9/10/2010
To Achieve the look and feel of a floating modal window required the use of an HTML tag called an iFrame. An iFrame is a separate window within the main browser window or HTML page This enables the ability to display two HTML pages within the same browser window This iFrame can be resized and moved around within the browser window
For each page with a Modal, the browser makes an additional request to the appserver The additional request happens at page load time for the iFrame This iFrame infrastructure is reused for all the Modal lookups on the same page
We have attempted to minimize the affect of this additional request from a performance perspective, however it may be questioned during load testing. You will see your http requests increase to support this feature and as a result more Jolt requests and some increase in web server CPU time This additional requests may increase the webserver and Appserver CPU time accordingly The impact of this enhancement is dependent upon the actual test that is performed during load testing
Scrollable Grids / Grids Zoom

Grids have been enhanced to allow for a much better user experience, but there is only a minor impact to performance, if any. Features include: One click Customization grids can be customized using 1 click instead of navigating to another page Grid Zoom allows the user to see more rows of data in a modal window than they could have seen on the actual page Grid zoom also allows for quicker data entry into the grid Scrollable grids allows more rows of data to be accessible to the user while not impacting the overall size of the page. Users can scroll through these rows while the column headers remain fixed
Mouse Over Pages

This features has been used across the 9.1 applications to aid in providing additional data to the user on specific fields. This page generally contains additional data pertaining to Vendor ID, Employee Info, Department ID etc. It Provides contextual data in an easy to access window on a transaction page without the need to navigate to another page. It also supplies additional data to the end user while reducing the number of clicks required to navigate within the application. These pages are loaded in the same appserver request as the main page, thus there is a minor performance overhead.
Deferred Mode vs. Interactive Mode

Ajax and the other features discussed here are considered Usability features and are not performance improvements. It is not recommended to change pages from deferred to interactive mode without considering the impact to performance as this will increase the load on your system.
50
Chapter 13 - Query Access Service (QAS)

QAS
QAS (Query Access Services) allows application developers to create web service-based applications that can retrieve data from PeopleSoft applications. QAS provides several web service operations. QAS utilizes the PeopleSoft Integration Broker framework to satisfy requests. When designing an application that will invoke QAS service operations, it is important to consider performance. In particular, there are three modes of query execution that each have particular performance characteristics: Synchronous Query Execution Asynchronous Query Execution Sync/Poll Query Execution
Synchronous Query Execution

When a synchronous query execution request is issued, the calling application process must wait for a reply. The full result of the query will be included in the reply. The amount of time that it takes to reply to the request depends on several factors, including but not limited to: Network performance Application server performance Database server performance Query complexity Amount of data in the result set
Note: Amount of data in the result set is determined by two factors: Number of rows of data Numbe of columns in each row
In general if you expect most queries to be simple in nature and result sets on the small size (e.g., hundreds of rows, not thousands of rows), then synchronous query execution may be an appropriate query execution mode. If you expect query complexity to be high and/or you expect that large result sets will be commonplace, or if response time is critical, then you may want to consider other modes of query execution.
Asynchronous Query Execution
51
9/10/2010
When an asynchronous query execution request is issued, the calling application process does not have to wait for the result set to be returned. When the result set is created, QAS will post to the calling application the result set. The calling application, of course, must be designed to be able to accept that post and process it accordingly.
The query result set resulting from an asynchronous query execution request is posted to the requesting application in one result post. For a large result set this means that the message posted to the calling application can become quite large. Thus, it is recommended that asynchronous query execution be used only when you expect query result sets to be small. Large result sets will take more time to convey in the network and take more time to process.
Sync/Poll Query Execution

When a sync/poll query execution request is issued, the return to the request includes a unique transaction id. Control returns very quickly to the calling process. At this point the calling application does not have any query results. It will have to issue additional web service calls to get the query results. After QAS receives the request it queues the request internally in the Integration Broker Pub/Sub system. The query will be run, and the query result set will be stored internally. When the calling program requests query results using the Get Query Results web service operation and passing the unique transaction id, query results will be returned in blocks, one block returned for each Get Query Results request. One query may result in one or more result blocks being passed. The maximum size of the result block can be configured.
This mode of execution is best suited for returning large or small numbers of rows of data. It is the mode typically used by reporting applications accessing PeopleSoft data via QAS.
Performance Considerations
Which execution model to run will depend on the number of rows and columns returned in the response. Keep the following guidelines in mind: Synchronous execution is not recommended for large result sets. Response data that consists of large number of columns with fewer rows of data has a slower response than response data which consists of fewer columns and more rows, even though the response size may be smaller. Requesting a query be executed in synch/poll mode with block size = 0 or Max will result in all data in one block. This is probably not what you want to do. In sync/poll execution, the larger the response block, the fewer blocks are returned and total response time for all blocks is less, but the user has to wait longer for the response. In synch/poll execution, if the heap size in the web server is not sufficient to retrieve the large response block, it will lead to a JAVA out of memory exception. In synch/poll execution, if a smaller block size (less than 100,000) is used, this will result in too many blocks containing very few rows. Optimal request block size in KB is in the range from 100,000 to 1,000,000.
52
Configuration Recommendations
Recommended configuration options include: Minimum Web Server Heap Size: Min & Max=512MB Client Heap Size: Min & Max=1024MB. Sync/Poll: Optimal Request Block Size in KB in the range from 100000 to 1000000. Sync/Poll: PSBRKHND instance count needs to be increased as response block count increases. Client SocketTimeout=600000 ms or Max limit approved for the query response time.
53
9/10/2010
Appendix B Revision History

Authors
Samuel To Michael Turner Michael Simons Ravi Selvaraj Indraneel Ganguli Dhinakaran Veeraraghavan David Nix
Reviewers
On Octocber 2007, the following people reviewed this White paper: Marc Varennes Ravi Shankar Colleen Murray Willie Suh Sunil Kulkarni Syamala Aramandla
Revision History
Date 2010-09-10 2010-06-23 Description Posted new version on 8.51 GA date although no content was changed. Modified file name. Updated formatting. Minor changes for browser performance and 8.50 GUI features. Added 8.50 Query Access Service (QAS) Added 8.50 content for GUI features. Copy edited and inserted numerous queries. Created document for PeopleTools 8.4 (inherited from 8.1 Performance RedPaper, version 8)
2010-06-22 2010-06-21 11/19/07 07/01/2002
54
Date 12/20/2002
Description Revised. Web Server Guidelines Updated service pack information for WebLogicUpdated BEA link, added workaround for Linux. Added section on load balancing Additional Guidelines - Added comment on compression and high-speed link.Rearranged and expanded section on Navigation Pages Caching. Added explanation on session timeout. Kernel Configurations Modified file descriptor limit for HP-UXAdded proc parameters for Tru64
01/02/2004
Revised: Application Server Guidelines Increased Recycle Count value.Added explanation on PSAPPSRV instance Web Server Guidelines Updated Execute Thread Count discussion. Added Jolt Request timing monitoring toolAdded WebSphere Resource Analyzer information Additional Guidelines Updated discussion about Compression Web Server Guidelines Updated with information about Oracle Application Server Call Telephony Interface Added new chapter PSAE and PSAESRV Added new chapter
11/04/2005
02/06/2006 05/23/2006 10/23/2006
Changed formatting Added missing step (set auditPwd) to the instructions to enable JOLT request timing trace Updated document with 8.48 information: Added information about preload cache, dynamic recycling and application server JVM options Added information on jolt session pooling, weighted load-balancing and dumping heap on OutOfMemoryError Added Integration Broker section Added Reporting Tools section Renamed Kernel Settings section to Operating System Settings section, and added Linux to the section
06/06/2007
Updated document with 8.49 information: Added information about Weblogic9.2 & Websphere 6.1.0.3 logging level settings to improve performance
55
9/10/2010
Date
Description Added information about to ensure JIT is enabled and class verification skipped Added information about tuning Asynch and Sync messaging on Integration Broker section Modified contents in PSAPPSRV instances, App server Dynamic recycle, Pre-load Cache, Parallel boot sections Added information about supported browsers in browser section
Copyright 2010, Oracle and/or its affiliates. All rights reserved. PeopleTools Performance Guidelines White Paper This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Copyright Oracle. Phone: +1.650.506.7000 Fax: +1.650.506.7200 oracle.com Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. AMD, All rights reserved. Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. Intel 56 and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. UNIX is a registered trademark licensed through X/Open Company, Ltd. 0110 document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission.

People Tools Performance Guidelines White Paper

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

People Tools Performance Guidelines White Paper

Uploaded by

Copyright:

Available Formats

An Oracle White Paper 10-September-2010

PeopleTools Performance Guidelines

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper 9/10/2010

STRUCTURE OF THIS WHITE PAPER

Copyright Oracle. All rights reserved.

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

Chapter 2 - Application Server Guidelines APPLICATION SERVER MEMORY GUIDELINES

Determining PeopleSoft AppServer Memory Usage Under NT

Copyright Oracle. All rights reserved.

Determining If Excessive Memory Swapping Happens Under NT

Determining Memory Usage on UNIX

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper Virtual memory size is:

Determining If Excessive Memory Swapping Occurs Under UNIX

APPLICATION SERVER RECYCLE COUNT

Copyright Oracle. All rights reserved.

APPLICATION SERVER DYNAMIC RECYCLE (PEOPLETOOLS 8.48 AND LATER)

SHARED CACHE FOR APPLICATION SERVER

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

PRELOAD CACHE (PEOPLETOOLS 8.48 AND LATER)

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

USE PARALLEL BOOT

SET APPLICATION SERVER JVM OPTIONS

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper 9/10/2010

USE JOLT SESSION POOLING (PEOPLETOOLS 8.48 AND LATER)

USE STRICT FAILOVER AND WEIGHTED LOAD-BALANCING (PEOPLETOOLS 8.48

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

ORACLE APPLICATION SERVER (ORACLE AS)

Capture JOLT Request Timing Traces

For more comprehensive data, repeat steps 4 through 7.

Modify Apache KeepAlive Setting

Modify ThreadsPerChild and Oc4jCacheSize

Disable HTTP Request Logging in mod_oc4j

Generate Heap Dumps on OutOfMemoryError

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

Log level Setting (PeopleTools 8.49 and Later)

Confirm That the Posix Performance Pack is Loaded

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper 9/10/2010

Set Thread Count

Confirm JRE Version

Set JVM Heap Size to 256MB or Higher

Set OS File Descriptor to 100*ThreadCount

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

Lower OS TCP/IP Cleanup/Timeout Settings

Monitor JVM Garbage Collection

Disable Servlet Reload

Copyright Oracle. All rights reserved.

Capture JOLT Request Timing Traces

For more comprehensive data, repeat steps 4 through 7.

Copyright Oracle. All rights reserved.

PeopleTools Performance Guidelines White Paper

Ensure That JIT is Enabled and Class Verification is Skipped

Setup and Configuration