Professional Documents
Culture Documents
Advice For The People Soft Oracle DB A
Advice For The People Soft Oracle DB A
Table of Contents
Introduction................................................................................................................................................2
Hardware and Operating System Support..................................................................................................2
Useful Resource....................................................................................................................................2
Oracle RDBMS Versions and Patchsets.....................................................................................................2
Supported Version Guidelines...............................................................................................................3
Oracle Limitations and Defects.............................................................................................................3
Certified Versions and Configurations..................................................................................................3
Initialization Parameters.............................................................................................................................4
"Required" Parameters..........................................................................................................................4
Parameters to be Avoided......................................................................................................................4
Parameters Often Confused...................................................................................................................5
Parameters Used as Customizations......................................................................................................5
Support of Non-standard Configurations..............................................................................................6
Useful KM Documents..........................................................................................................................6
Statistics Management...............................................................................................................................6
Oracle Features Critical to Statistics Management...............................................................................6
Recommendations Regarding Managing Statistics...............................................................................8
Useful KM Documents..........................................................................................................................8
Indexing Techniques................................................................................................................................10
Useful KM Documents........................................................................................................................11
Optimizer Limitations..............................................................................................................................11
Useful Tools & References.......................................................................................................................11
SQLTXPLAIN (SQLT)........................................................................................................................11
pscbo_stats...........................................................................................................................................11
AWR / StatsPack Report.....................................................................................................................12
PSPRCSRQST Batch Trigger.............................................................................................................12
OS Watcher..........................................................................................................................................12
Remote Diagnostic Agent (RDA)........................................................................................................12
ORAchk Health Checks for the Oracle Stack (ORAchk)...................................................................12
How to Configure PeopleSoft for Maximum Availability (MAA).....................................................12
Updates and Suggestions.........................................................................................................................13
20140508-AdviceForThePeopleSoftOracleDBA.doc 1 of 13
Introduction
This document is intended the Oracle DBA who seeks a "big picture understanding" regarding how to
support a PeopleSoft Application. It is a brief overview that contains advice from PeopleSoft Support
Install/Upgrade and CoE teams with links to relevant KM documents.
It is organized as a series of steps starting with comments related to hardware, and working "up" from
there, e.g. from hardware, operating systems, database versions and patchsets, initialization settings,
statistics, and so on.
This documented was written with the anticipation that the reader is an Oracle DBA that is skilled in
the various aspects of implementation, administration, and tuning of the Oracle RDBMS. Additionally,
it will be helpful for the reader to be fluent in PeopleSoft technologies and/or have access to PeopleSoft
technical staff that can modify settings to meet specific implementation needs.
The information contained in this document applies to Oracle databases used for PeopleSoft
Applications built on PeopleTools 8.50+, specifically on Oracle 10g, through 12c.
Useful Resource
• Certifications Tab on My Oracle Support
20140508-AdviceForThePeopleSoftOracleDBA.doc 2 of 13
Supported Version Guidelines
A given PeopleTools release is certified on Oracle database release of a specific minimum patchset.
Within an Oracle release, that oneand all later patchsets are supported as well, unless otherwise
indicated.
For example, PeopleTools 8.51 is initially certified on Oracle releases 10gR2, 11gR1, and 11gR2 -
patchsets 10.2.0.4, 11.1.0.7, and 11.2.0.1, respectively. All patchsets to each release after those certified
patchsets are supported as well.
But this broad support of the Oracle RDBMS certainly does not imply that all PeopleSoft Applications
will perform equally well on all versions of the Oracle RDBMS. Some characteristics of a given release
or patchset can impact the PeopleSoft Application significantly. PeopleTools Support encourages
customers to use patchsets that are as current as practical for a given release.
20140508-AdviceForThePeopleSoftOracleDBA.doc 3 of 13
Installation PeopleTools - Master List" (Doc ID 756571.1). Note: ensure that the versions of the
various components being implemented "match" and are supported before contacting Oracle
Support.
Initialization Parameters
Initialization parameters are critical to the decisions that the Oracle optimizer makes. This section
describes some of the more significant settings and their impact on a PeopleSoft Application.
"Required" Parameters
Related to currently supported Oracle versions, three initialization parameters have been indicated as
"required" by PeopleTools Development. One has since been changed to a privilege requirement. This
section highlights these parameters and briefly explains how they came to be required.
• _gby_hash_aggregation_enabled=false - Oracle 10g implemented a high-performance, hash-
based sorting algorithm for use when processing DISTINCT clauses. This new algorithm does
not guarantee that the rows will be ordered whereas the previous algorithm did sort the rows.
Many PeopleSoft applications were written with the expectation of the sorted rows and will not
behave properly without this sorting. Until it can be established that all PeopleSoft
Applications have been rewritten to address this change in behavior, PeopleTools requires this
new algorithm to be disabled to remediate this side effect. For more information, refer to the
KM document listed in the following section entitled "Useful KM Documents."
• _unnest_subquery=false - During internal testing with the default value in place, performance
issues occurred in on-line processing. There were also some occasional errors creating views.
To minimize the risks of encountering these issues, the default value for this parameter needs to
be FALSE. There is some limited flexibility in the use of this parameter explained later in this
documented.
• o7_dictionary_accessibility - This parameter was initially added to allow functionality needed
for the triggers implemented by PeopleSoft in PeopleTools 8.48 to support PeopleSoft mobile
applications framework. In May 2010, the requirement to use this parameter was eliminated and
superseded with a requirement to implement two GRANTs. See the document "Required
Interim Patches for the Oracle Database with PeopleSoft." listed below.
Additional parameters are required for some "special" cases. These settings are found in the Enterprise
PeopleTools Installation Guides. For example:
• nls_length_semantics=char. Required for Unicode databases in some versions. See Installation
Guide, Section "Preparing for Installation" for additional details.
Parameters to be Avoided
It is not unusual to see certain settings creep into a PeopleSoft system over time, typically as the result
of incremental workarounds to address performance issues or perceived optimizer limitations. This
section will provide a brief explanation regarding certain settings that should, as a rule, be avoided.
The following three parameters heavily skew the CBO's calculations. With rare exceptions - typically
only as a workaround to a CBO defect or design limitation - we recommend that only the default values
should be used for these parameters.
20140508-AdviceForThePeopleSoftOracleDBA.doc 4 of 13
• optimizer_index_cost_adj - default: 100 This setting, and the next, have been traditionally been
used in much earlier versions of Oracle RDBMS to globally coerce the CBO to prefer the use of
indexes. The continued global use of this parameter in current versions of Oracle is not
recommended since it can have a profoundly negative impact on the overall performance
stability of the database.
• optimizer_index_caching - default: 0 See comments immediately above.
• db_file_multiblock_read_count - default: unset, dynamically calculated by the RDBMS.
• "underscore" parameters: The use of underscore parameters should be avoided unless at the
specific direction of Oracle Server Tech Support for the purpose of addressing a specific bug.
Care must be taken to remove them as soon as the database is upgraded to a version/patchset
that addresses the bug. Other than those listed previously, PeopleSoft Support strongly
discourages the use of any underscore parameters.
• cursor_sharing: The cursor_sharing parameter deserves special attention as its global (mis)use
can have a deleterious impact on system stability in a PeopleSoft Application. The default
setting of cursor_sharing is exact and is the only value should be used with PeopleSoft
Applications.
Note: The setting of cursor_sharing=similar has been deprecated from Oracle and should not
be used in any context for PeopleSoft Applications.
If there is a specific use case where the use of the query unnesting feature is necessary, enabling
this parameter is allowed, i.e. as either a hint or session setting. The use of this parameter will
20140508-AdviceForThePeopleSoftOracleDBA.doc 5 of 13
be supported as a customization, and may need to be be removed during the diagnostic process
of a Service Request.
Useful KM Documents
• From 10gR2, HASH UNIQUE Operation Returns Results in UNSORTED ORDER by Default
(Doc ID 341838.1)
• Required Interim Patches for the Oracle Database with PeopleSoft (KM Doc ID 1100831.1)
• Operating System, RDBMS & Additional Component Patches Required for Installation
PeopleTools - Master List (KM Doc ID 756571.1)
• ANNOUNCEMENT: Deprecating the cursor_sharing = 'SIMILAR' setting. (KM document:
1169017.1)
Statistics Management
Traditionally, PeopleTools Development has made suggestions to assist in the gathering of statistics,
e.g. in Red Papers and default configurations. Decisions regarding statistics gathering have been left to
the customer with the assumption that defaults would both be used and be adequate.
In spite of the maturity of the Oracle CBO, some characteristics have emerged over time that make the
use of "defaults" somewhat problematic for PeopleSoft Applications. This section will explore the more
significant anomalies and then provide recommendations to help mitigate their impact.
In Oracle10g, the default sample size was usually too small to be effective. To that end, Oracle Support
recommends that, in Oracle10g, the sample size be set to 100% whenever practical, or based on the
following schedule that is table size-dependent:
100% for tables < 1M rows
30% for tables up to 10M rows
10% for tables up to 100M rows
3% for tables up to 1B rows
1% for tables > 1B rows
In Oracle 11g, the automatic sampling algorithm was so completely changed that the use of
20140508-AdviceForThePeopleSoftOracleDBA.doc 6 of 13
AUTO_SAMPLE_SIZE is now appropriate and recommended. Studies have shown that the quality of
statistics gathered approaches that of a 99% sample size yet provides, the speed of a 10% sample size.
One exception to the use of AUTO_SAMPLE_SIZE exists when a column has histograms and the data
in the column is very skewed. In that case, it may be better to explicitly collect with a large sample size.
To summarize, the recommended sample size to be used is as follows:
• In Oracle 11g, use AUTO_SAMPLE_SIZE as the rule.
• In Oracle 10g, avoid the use of the AUTO_SAMPLE_SIZE in all cases. Use a sample size of
100% where practical. Where size prohibits a 100% sample, use the schedule above to improve
performance.
Histograms
With the evolution of the Oracle 11g optimizer, histograms have more broadly become accepted as
beneficial and their use is now recommended to enhance plan stability and performance. When
gathering statistics on Oracle 11g or later, using a METHOD_OPT of AUTO is appropriate.
As a rule, avoid the default use of histograms on Oracle 10g. Use them only in specific situations where
it is known that they address a specific column and there is no side effect with their use. When
gathering statistics on Oracle 10g, use a METHOD_OPT that includes 'SIZE 1', which will prevent the
generation of histograms.
Automatic gather_stats Job
By default, Oracle database are delivered with an auto_stats job that schedules the
gather_database_stats procedure. Because of the peculiarities of PeopleSoft applications running on
Oracle, some problems arise with this procedure and, by extension, the job.
• Histograms: The procedure will automatically generate histograms based on whether a column
has been used as a predicate in SQL statement history. As mentioned earlier, this is problematic
in Oracle 10g. See the previous section regarding histograms.
• Sample Size: The default sample size used by the procedure is AUTO. Unfortunately, in
Oracle 10g, this value produces a sample size that is so small that the results are often
inaccurate. Increasing the sample size in Oracle 10g can dramatically improve the quality of the
statistics, but impact the performance of the process. (Note: The AUTO_SAMPLE_SIZE is
Oracle 11g works correctly and is the recommended value for most samplings.)
• Impact on Dynamic Sampling: Lastly, there is insufficient control over what tables should not
have statistics gathered. When statistics are deleted to invoke dynamic sampling, that table must
be locked to signal the job not to gather the statistics on that table. But if the statistics are
locked, it will likely cause PeopleSoft batch processes to ABEND when an update of the
statistics is attempted from the program.
Delayed Invalidation of Statistics
There is a delay that occurs after the gathering of statistics and the invalidation of plans. The delivered
syntax from PeopleSoft does not override this delay which is the cause of some unpredictable run time
behavior. It is recommended that NO_INVALIDATE be set to FALSE when gathering statistics.
20140508-AdviceForThePeopleSoftOracleDBA.doc 7 of 13
Recommendations Regarding Managing Statistics
There are few recommendations worth considering to address the most common statistics-related issues
with PeopleSoft Applications.
Modify %UpdateStats metaSQL
Useful KM Documents
• E-ORA How to alter the database preference for NO_INVALIDATE for PeopleTools
applications (Doc ID 1537099.1)
Implement pscbo_stats
pscbo_stats is a joint venture between PeopleSoft CoE and Oracle Server Tech CoE and is a tool for
implementing dynamic sampling as well as embracing the preferred settings outlined in this document.
It provides a consistent, reliable way to maintain database statistics for PeopleSoft Applications. It
includes methods to gathering stats at either the schema- or table-level, and automatically implements
dynamic sampling for known volatile tables, e.g. temporary storage tables in Application Engine and
COBOL.
Useful KM Document:
• KM document "pscbo_stats - Improving Statistics in Oracle RDBMS for PeopleSoft Enterprise
(Doc ID 1322888.1)"
Gather data dictionary statistics
The PeopleSoft Application database has many objects, numbering well in the tens-of-thousands. Each
of those objects is defined in the Oracle data dictionary, a set of tables that the optimizer is constantly
querying to validate the existence of tables, columns, indexes, constraints, etc.
Neglecting to update the statistics on this data dictionary following major DDL changes is a common
oversight. After building, or upgrading, a PeopleSoft database, it is wise to gather the Oracle dictionary
statistics.
20140508-AdviceForThePeopleSoftOracleDBA.doc 8 of 13
The following example script shows the necessary syntax, e.g. (run as SYSDBA):
SET SERVEROUT ON TRIMS ON LINES 1000;
SPO gather_dict_stats.log;
exec dbms_stats.gather_dictionary_stats(estimate_percent=>100,
method_opt=>’FOR ALL COLUMNS SIZE 1’);
QUIT;
Current workload statistics can be very useful for the optimizer (especially as of Oracle 11g), but are
quite often overlooked since the defaults often work quite well. If you have reason to believe that the
default values are inadequate, or if for some reason the behavior of your system changes from time to
time, gathering workload statistics may be appropriate.
If you choose to gather the workload statistics, be sure to refresh them after any major RDBMS
upgrades or changes, or when significant hardware or OS changes are performed. Typically, the data is
collected during a one-to-two hour window of "normal" system activity, but that sample varies
depending on several implementation factors.
Re-gather workload statistics each time database usage patterns changes significantly, e.g. after a
database upgrade or when there is significant change on the System affecting CPU or IO metrics. As
with any global change, manage the risk associated with regressions by performing adequate testing.
Since this data is being collected by taking two snap shots of the metadata during "normal" system
activity, the remote possibility for an observable performance impact exists. This impact is temporary
and is usually quite small. The benefits of having valid workload statistics normally outweigh the
temporary costs of gathering them.
The command to start the baseline sample is:
exec DBMS_STATS.GATHER_SYSTEM_STATS('START');
Is is extremely important to review the results after gathering workload system statistics. Confirm all
values are within reasonable ranges with the following sample script (run as SYSDBA):
SET SERVEROUT ON TRIMS ON LINES 1000;
SPO display_workload_stats.log;
SELECT pname, pval1
FROM sys.aux_stats$
WHERE sname = 'SYSSTATS_MAIN';
QUIT;
For example, the SREADTIM are typically 10 or less, i.e. 10ms or less. The MREADTIM is typically
50 or less, i.e. 50ms or less. If they are not, more specifically if they happen to be 10,000 times that
value, you may be experiencing the bug described in Note 9842771.8.
Useful KM Document:
• Bug 9842771 - Wrong SREADTIM and MREADTIM statistics in AUX_STATS$ (Doc ID
9842771.8)
20140508-AdviceForThePeopleSoftOracleDBA.doc 9 of 13
Indexing Techniques
Indexing strategies abound and are, in many ways, an art form for the skilled application tuner. While
improving an individual access path is somewhat mechanical, building an index that will assist several
access paths through various data sets without interfering with application performance can require
significant skill. These trade offs should always be made in an informed manner.
While this section does not intend to exhaustively advise how to perform index tuning, it does include
some valuable "lessons learned" for the DBA doing index tuning for PeopleSoft Applications.
• Index Uniqueness - PeopleSoft Applications use unique indexes to enforce identity integrity.
Although re-ordering the keys in a unique index will not impact the integrity of a PeopleSoft
Application, it is imperative that the presence of the keys in a unique index not be altered in any
way. Generally, modifying the unique indexes is not needed, even for performance reasons.
• Altering or adding non-unique indexes - Non-unique indexes are algorithmically built by
PeopleTools Application Designer based on the application developers' choices when defining
record structures. As such, they represent an "educated guess" regarding how data will be
accessed.
Other than for performance reasons, there is nothing inherent in the PeopleSoft Application
requiring these indexes to remain, or that they remain, to be unmodified. Remember that
PeopleTools maintains its own data dictionary, distinct from the Oracle data dictionary, that
needs to be updated with any changes made to indexes so that those changes can be preserved
during an Application upgrade.
• Function-Based Indexes - The DESC index is an example of a Function-Based Index that may
add enough optimizer complexity and storage requirements that its benefits are offset. With
Oracle 12c databases, PeopleTools requires the disabling of DESC indexes, and encourages it in
previous versions of the database. For details, see the KM documented listed at the end of this
section.
• Equi-join predicates should precede scanned predicates in indexes - "Best practices" used to
suggest that highly selective values should always lead an index. But that is not always true, and
in a PeopleSoft Application where range scanning is common, this misunderstanding can cause
significant performance issues.
When an index is added to support a query where a range scan occurs, it is important to locate
the scanned key "later" in the index order. While it is true that subsequent keys can be used as
filter predicates, the gains of being used as an access predicate are ended if the key appears to
"early" in the index. To illustrate, consider the following simple query:
SELECT COL_A, COL_B, COL_C
FROM MYTABLE
WHERE COL_A=:1 AND COL_B =:2 AND COL_C > 100;
For the purposes of this example, assume that all three columns are required to be reasonably
selective and that the optimizer "wants" to use an index to retrieve this data. Also assume COL_C
is the most selective and COL_A is the least selective. It would be tempting to add the following
index to support this query:
MYTABLE(COL_C, COL_B, COL_A)
20140508-AdviceForThePeopleSoftOracleDBA.doc 10 of 13
But that arrangement would be inefficient for this query because the range scan on COL_C
would prevent the COL_B or COL_A from being used as access predicates, though they might
be used as filter predicates. A better arrangement of these keys would be one of the following:
PS_MYTABLE(COL_A, COL_B, COL_C)
PS_MYTABLE(COL_B, COL_A, COL_C)
The decision regarding which key should come first could be informed by whether other queries
use only COL_A or COL_B.
Useful KM Documents
• Oracle 12c and the Removal of Descending Indices for PeopleSoft Databases (Doc ID
1576243.1)
Optimizer Limitations
There are a few limitations that can cause issues and should be avoided if possible.
• Avoid the use of BOTH histograms AND bind peeking in Oracle 10g - The combination of these
features tend to produce plan instability in Oracle 10g. PeopleSoft relies heavily on the use of
bind variables, so avoiding histograms is a reasonable approach to avoid this problematic
combination.
• Avoid range scans on "padded" fields of type VARCHAR - When the VARCHAR field is used in
a BETWEEN clause, the CBO can make assumptions that are not adequate, most notably seen
in cases where the length of the data is large and starts with, or is "padded" with, the same
several characters, e.g. 00000123. If this type of data is used in an implementation, additional
care will be needed to ensure that the optimizer has enough information to generate efficient
plans.
SQLTXPLAIN (SQLT)
SQLT is a powerful reporting tool that collects data related to the execution of a given piece of SQL. It
conveniently collects execution plans, CBO statistics, metadata, performance statistics, configuration
parameters, etc. into a single archive that can easily be attached to an SR.
• SQLT - Tool That Helps To Diagnose SQL Statements Performing Poorly (Doc ID 215187.1)
pscbo_stats
pscbo_stats is a joint venture between PeopleSoft CoE and Oracle Server Tech CoE and is a tool for
implementing dynamic sampling as well as embracing the preferred settings outlined in this document.
It provides a consistent, reliable way to maintain database statistics for PeopleSoft Applications. It
includes methods to gathering stats at either the schema- or table-level, and automatically implements
dynamic sampling for known volatile tables, e.g. temporary storage tables in Application Engine and
COBOL.
20140508-AdviceForThePeopleSoftOracleDBA.doc 11 of 13
• pscbo_stats - Improving Statistics in Oracle RDBMS for PeopleSoft Enterprise (Doc ID
1322888.1)
OS Watcher
OS Watcher (OSW) is a collection of UNIX shell scripts intended to collect and archive operating
system and network metrics to aid support in diagnosing performance issues. OSW operates as a set of
background processes on the server and gathers OS data on a regular basis, invoking such Unix utilities
as vmstat, netstat and iostat.
• OS Watcher User Guide (Doc ID 301137.1)
20140508-AdviceForThePeopleSoftOracleDBA.doc 12 of 13
enhancements in PeopleSoft enable faster application failover and reporting offloading to Active Data
Guard.
• E-ORA How to Configure PeopleSoft for Maximum Availability (Doc ID 1446793.1)
20140508-AdviceForThePeopleSoftOracleDBA.doc 13 of 13