Professional Documents
Culture Documents
8/2007
PURPOSE STATEMENT
This document is intended for information purposes only, and may not be incorporated
into any contract. It is not a commitment to deliver any material, code, or functionality,
and should not be relied upon in making purchasing decisions. The development,
release, and timing of any features or functionality described for Oracle’s products
remains at the sole discretion of Oracle. Updates and enhancements are provided in
accordance with Oracle’s Technical Support Policies at:
www.oracle.com/support/collateral/oracle-technical-support-policies.pdf
UBE basics............................................................................................................................................ 1
General considerations for long running UBEs ......................................................................... 1
Concurrency.......................................................................................................................................... 2
Pre-Payroll UBE (R07200)............................................................................................................. 3
Sales Order Update (R42800)........................................................................................................ 3
Material Requirements Planning (R3482 / R3483) .................................................................... 4
Troubleshooting tips for MRP Performance issues: ............................................................. 4
UBE QUEUES .................................................................................................................................... 6
Multi-CPU machines....................................................................................................................... 6
Parallelism, concurrency, and multiple CPUs......................................................................... 6
Multi-threaded Queues ................................................................................................................... 7
UBE Subsystems.................................................................................................................................. 7
Platform Specifics.......................................................................................................................... 10
iSeries Memory Pools............................................................................................................... 10
Configuration...................................................................................................................................... 11
iSeries SQL packages .................................................................................................................... 12
Oracle Update Statistics................................................................................................................ 12
Important JDE.INI settings related to UBEs........................................................................... 13
Data Close to Processing ............................................................................................................. 13
The key issues:........................................................................................................................... 13
More details on data on iSeries, processing on other platform ..................................... 15
Qaqqini ........................................................................................................................................... 15
UBE logging and debugging ............................................................................................................ 16
Skewed extrapolations .................................................................................................................. 18
Last but not least: the ‘debug’ factor…...................................................................................... 19
UBE BASICS
This paper is intended to provide guidance on improving the runtime and throughput of EnterpriseOne batch
processes (UBE), primarily via configuration techniques and use case methodology. Performance items related to
coding and design of UBEs are not addressed here.
Oracle JD Edwards EnterpriseOne UBEs differ from interactive applications in a number of important ways:
A UBE is a distinct and separate process which ends when the UBE ends.
(Subsystem UBEs function differently; see that section)
Call Object Kernels, on the other hand, run a particular business function then continue to process,
waiting for the next incoming request from an interactive program to run a business function. They
keep running until the Enterprise Server is stopped or until ab-ending.
A UBE runs only one thread of execution for one user’s UBE, while a call object kernel runs business functions
for many users, possibly in a true multi-threaded architecture.
In terms of CPU usage, UBEs are like lap swimmers, while the CPUs are like lanes in the pool. One UBE wants
one CPU, and wants it for the duration of its run. It does not “share” well with other processes.
Call Object kernels, conversely, are like children jumping in and out of the pool. A Call Object Kernel uses the
CPU in spurts, but does not generally monopolize it for extended periods.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 1
Be conscious of business cycles and time windows when large UBE jobs are run.
Plan to avoid running large UBEs when heavy interactive processing is running.
Run large UBEs during downtimes (such as nights or weekends).
CONCURRENCY
The single biggest impact item for increasing UBE throughput – and perhaps the most underutilized - is
concurrency.
Many commonly used UBEs are capable of being broken up by data selection into multiple versions which are run
simultaneously. It is crucial that the data selections of the individual versions be non-overlapping to avoid data
corruption or data inconsistency.
These versions can then be run concurrently in a queue capable of running multiple UBEs.
Important: Any long running UBE, even if it’s not mentioned in this document, should be viewed as a
possible candidate for concurrency.
Important: If a server exists with multiple CPUs, try running more than one job to achieve substantial
throughput increases.
Multiprocessor servers are ubiquitous, especially within Oracle JD Edwards EnterpriseOne implementations, but
few programs ever utilize more than one processor at a time.
Running UBEs concurrently will make use of your multi-CPU hardware to maximize throughput. Multiple CPUs
do not make a job run faster; they allow more jobs to be run at the same time. See the section on UBE Queues for
more details.
Some common UBEs are routinely run concurrently:
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 2
Common Concurrent UBEs
UBE Suite Additional Information
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 3
Please refer to JD Edwards EnterpriseOne Sales Order Management 8.12 Implementation Guide, "Setting Processing
Options for Update Customer Sales (R42800)" for more information.
Once the table F986113 is loaded, each subsystem will read a record from F986113 and plan the item. So if the
user is running parallel processing with 3 subsystems, then each subsystem will read a record from F986113.
Subsystem 1 will plan item A (LLX=1) , subsystem 2 will plan item B (LLX = 1) and subsystem 3 will plan item C
(LLX=1). The UBE ensures that all items which are getting processed by each subsystem belong to same low level
code. Once all the items at the current low level code (LLX=1) are processed, the UBE deletes the F986113
records and increments the low level code value to 2 (LLX=2). This process is repeated until highest low level
code is planned. After all the records are processed, the subsystem jobs are terminated.
One customer reduced a 15 hour serial MRP run to a 4-hour parallel run by using the subsystem-based parallel
processing.
See the section "UBE Subsystems" for other uses of subsystem jobs.
Please refer to JD Edwards EnterpriseOne Tools 8.96 Development Tools: Report Design Aid Guide, "Working with
Subsystem Jobs" for more information.
Certain steps can be followed to increase the performance of MRP batch processes:
1. Set the planning horizon in R3482/R3483 to a value only as long as the business requires.
(See related solution 200782880 R3482/R3483 processing options).
2. Make sure the planning fence (F4102) is set to a maximum value no greater than the horizon.
3. Change the data selection and processing options. Carefully choose the data selection and set the appropriate
values for the processing options (PO) in order to cut down on unneeded processing. Narrow down the data
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 4
being processed through data selection. Several POs affect R3483 performance. Set the appropriate values
according to your needs.
a. R3482/3 performance tab
i. PO 2 should be blank if not using the print code.
ii. PO 3 should be blank, unless there is a special business need to have messages and time
series records for Phantoms.
iii. PO 4 can limit the number of work orders (WO’s) processed by using the status.
iv. PO 5 should not be set to “1”, unless you are using rate schedules.
b. R3483 multi-facility tab
i. PO 1 Process Each Branch should be blank.
ii. PO 2 should include a Date Branch to prevent multiple hits to the shop floor calendar. (See
related solution: 200782880 R3482/R3483 Processing Options).
4. Set the variable INITIAL_BRANCH_COUNT in BSFN B3401270.h when running R3483. The value could
affect performance. It is set to a default value of 50, but can be changed to the maximum number of branch
plants you are processing. This ensures that memory is not allocated for more branches than you are actually
processing.
5. Delete the Supply/Demand inclusion rules that are not being used.
6. Creating your own quantity type UDC (34/QT) may help MRP processing time by reducing the number of
calculations made by R3483. The F3413 Time Series Table has by far the greatest number of records written by
R3483. Elimination of some of those records that you will not use may help MRP Performance.
The standard MRP 34/QT Quantity Type table contains many quantity types calculated by R3483, and some
business models may not need all the quantity types provided in the standard table.
One possibility is to create a table with the relevant quantity types that you will be using. For example, if you
are not using Repetitive Manufacturing, or Rate Schedules, then eliminate the quantity types for Kanban, and
Rate Schedules. There are limitations in using the ATP and CATP lines, since MRP is a snapshot, and in a
dynamic environment, will only be valid at the end of the MRP run. Therefore, you might consider eliminating
those quantity types as well.
Important: Do not modify the standard 34/QT table. Create your own UDC table, copy the 34/QT, and then
delete the quantity types that you do not need. Then in the processing option in R3483 calling for quantity type table
(now populated with QT) enter your new value. MRP will then make the calculations for the quantity types relevant
only to your business model, and save valuable processing time.
7. Purge the F4801 table of closed work orders and the F4211 table of closed sales orders.
8. Contact GSC to receive a custom code change document to suppress the generation of the time series if the
time series is not used in your business model. This is will significantly improve performance.
9. Call GSC to receive a custom code change to avoid table caching of the F4101, F4102 and F4801 tables. This
should be done only if the MRP program is running out of memory, as this will actually reduce throughput.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 5
UBE QUEUES
UBE queues are created and configured via the P986130 application. Their purpose is to control the total number
of simultaneous UBEs running on the system and to maintain data integrity by controlling which UBEs run
concurrently.
Queues can be configured so that only one UBE jobs can be running at a given time, or so that multiple jobs can
be running at the same time. A common misconception about UBE throughput is the notion that: “We’ll just add
more queues and things will run faster.” However - the number of concurrent UBEs which can be running on a system
is limited by CPUs.
The rule of thumb is one CPU will support 1-2 UBEs.
On newer processors, such as the Power 5 iSeries processor, this could be a bit higher. Refer to the following link
to an IBM whitepaper with UBE benchmark results on a Power5 iSeries processor:
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100511
For example, if you have a 2 CPU machine, you cannot run 20 concurrent UBEs just because you set up queues
with a max concurrency of 20. That is like saying: “My bank account can’t be overdrawn…I still have more checks.” You
must have the CPU capacity to support the potential number of concurrent UBEs in the queues you have created.
This is often a badly overlooked fact in machine sizing effort leading to unpleasant surprises in real machine
capacity come go-live.
Important: The single biggest limiting factor in sizing an E1 Enterprise server which is most commonly
overlooked is the number of concurrent UBEs.
Multi-CPU machines
Another misconception is in the impact of multi-CPU machines.
Important: A single UBE job will not run any faster on an 8-CPU machine than on a one-CPU machine.
This is because one UBE job (in general) will utilize only one CPU.
There are many misconceptions about parallelism, concurrency, and multiple CPUs. A multi-CPU machine is to a
single-CPU machine as a multi-lane road is to a single-lane road. A single car traveling on a multi-lane road cannot
carry any more passengers than when on a single-lane road.
A multi-lane road will allow more of the same cars to travel at the same time.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 6
Important: Software will not run faster on a multi-CPU machine unless you use the CPUs.
Multiple CPUs do not cause a single UBE to run faster; they allow more UBEs to be run simultaneously.
Multi-threaded Queues
The term "multi-threaded queue" is a commonly-used moniker but is inaccurate. UBEs are single-threaded
processes. They are not multi-threaded and neither are UBE queues. The term "threaded" should never be used in
association with UBEs. When the term “multi-threaded queue” is used, it means that more than one UBE job can
be running in the queue at a given time. “Concurrent queue” is a more accurate name.
This spurious naming convention leads to confusion between UBE processing and Business Function processing.
Call Object kernel jobs, starting in Tools Release 8.95 (8.96 on iSeries), are true multi-threaded processes - or
rather they can be configured that way. Multi-threaded call object kernels allow more than one business function to
run in a call object kernel at a time. It does this via threading within the kernel job. This helps eliminate processing
delays which can occur in business functions called from interactive applications.
UBEs processing is completely and totally unrelated to call object kernels. Call Object Kernels are used for running
interactive business functions and do not impact the performance of UBEs in any way, shape, or form. UBEs do
not make use of call object kernels to run Business functions. They call BSFNs in place, within the UBE process.
Note: One common problem in queue design is that short UBEs get stuck in a “single threaded” queue behind a
UBE which takes hours. A solution to this issue is to create dedicated queues for short jobs and other queues for
longer jobs. This allows short running jobs to process quickly and move out of the queue, allowing other jobs to
process.
UBE SUBSYSTEMS
Subsystems were mentioned previously in the context of the MRP parallel processing solution, but Oracle JD
Edwards EnterpriseOne subsystem UBEs in general have different purposes. A subsystem in EnterpriseOne is a
special type of UBE report. The subsystem table F986113 contains the following information:
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 7
• Report Name
• Report Version
• Report Interconnect values
The Work with Subsystems application (P986113) is used to start subsystem jobs
A subsystem report, once started, will loop continuously reading records from F986113 until explicitly stopped.
Unlike a non-subsystem UBE, it does not end when the UBE processing ends.
• A new record in F986113 indicates a new subsystem report to be run.
• Subsystems are server only –subsystem job cannot be run on the client.
Subsystem
Load Specs / Init Is Subsystem is still Active?
Yes No
No- Subsystem
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 8
o User needs a “one-click” solution to run the report.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 9
R46171 XJDE0007 Warehouse Subsystem Processing
R47011 XJDE0002 EDI Inbound Purchase Order Edit/Update - Subsystem
R47016ZO XJDE0002 Outbound POE Processor Subsystem
R470412 XJDE0002 EDI Inbound Invoice/Match (Subsystem)
R47071 XJDE0002 EDI Inbound Edit/Update-Subsystem
R47121 XJDE0002 Product Activity Data Edit/Create Subsystem
R47127O XJDE0002 Outbound Prod. Act. Data Subsystem-Update As Sent
R4942Z1I ZJDE0002 Shipment Creation - Subsystem Version
Reports which run in 1-2 minutes have been reduced to 10-20 seconds when the subsystem version is used
Platform Specifics
The following recommendations are specific to the platform that Oracle JD Edwards EnterpriseOne is run on.
Important: You should assign a pool for use by UBEs only. Do not assign any other non-UBE work to this
pool.
See the previous analogy where UBEs are compared to “lap swimmers”, while others jobs continuously jump in
and out of the pool. This is especially true of the memory pool on iSeries.
If the UBE is memory limited, more memory will result in a faster job. If there is a high degree of paging in the
UBE pool exhibited in Work With User Jobs (see the picture below), then the UBE would likely benefit from
increased pool size
UBE jobs want to use about 500K per job – as a guideline. Some jobs will use much more memory than this, some
will use less.
Some actual examples of improvements seen a customer for a memory-constrained UBE include:
With a 1.5 GB UBE pool: 26k records per hour was achieved
With a 3.5 GB UBE pool: 39k records per hour was achieved
Below is another example of how UBE runtime was reduced by increasing the UBE memory pool. This was for a
UBE run with a high degree of concurrency and upwards of 10 concurrent jobs:
6 GB pool: 81 minutes
10 GB pool: 54 minutes
20 GB pool: 20 minutes (Optimal size)
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 10
Note
Memory Paging
Pools Stats
CONFIGURATION
Some common “low-hanging fruit” items for gaining throughput from UBEs
Data selection – Reduce the amount of rows processed to prevent a UBE from running many unnecessary
iterations.
Processing Options - are often many and complex. Some UBEs, such as Sales Order Update (R42800), have
upwards of 100 separate options.
One simple setting can add lots of processing to the program.
Review every single one as one of the first steps in diagnosing UBEs which are running more slowly
than desired.
Determine that the UBE is not performing unneeded calculations or excess functionality as a result of
unneeded setting.
It is crucial to know your application thoroughly
Some examples of features which may not be needed:
Advanced Pricing
Advanced Cost Accounting
Advanced Preferences
Flex Accounting
Transportation
Purging and archiving of large transactional tables -- many of the larger tables should have their content
archived to history as often as business needs will allow. Large tables with unneeded historical data lead to longer
search and insert times for these tables.
Some examples:
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 11
F4801 – Work Order Master File
R01131P - Purge Completed Tasks UBE
F4211 – Sales Order Entry
F42119 – Sales Order History
F0911 – General Ledger
F0911P – General Ledger History
Note: The appearance of strange SQL errors in the jde.log may be an indication of package corruption.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 12
Performance improvements will vary and depend on a wide number of variables including table size, the query
being run, and when statistics were last updated.
This is for iSeries only. If this is set to ‘2’, then you change it to ‘1’ or a ‘0’ for daily use. A value of ‘2’ is only
intended for running Oracle JD Edwards EnterpriseOne table conversion UBEs (TCs). It is common for this to
be overlooked following installations or upgrades. A value of ‘2’ will not cause functional problems, but UBEs will
run more slowly since SQL Packages must be recreated each time a given UBE is run. Slower performance will be
noticed most particularly on shorter-running UBEs. Changing this value requires deleting packages and re-starting
the Oracle JD Edwards EnterpriseOne services.
Here are the meanings and performance implications of the settings in 8.9 and beyond:
• SQL Package Library=2. When TCs need to be run, normal UBEs will run slower since the SQL packages
do not get reused. They will be re-built each time the UBE is run, and are named for the job number (for
example, T123456). Maintenance is required on a recycling of EnterpriseOne Services by deleting SQL
packages using the commands below:
o DLTSQLPKG QRECOVERY/T*
o DLTSQLPKG QRECOVERY/OW*
• SQL Package Library=1. UBE packages will be named “Rnnnnn” and all UBEs will reuse their SQL packages. You
will need to delete any old UBE SQL packages in QRECOVERY using the following command:
o DLTSQLPKG QRECOVERY/R*
Note: No UBEs can be running on the server while the SQL packages are being deleted.
• SQL Package Library=0. Puts Kernel and UBE SQL Packages in the System Library such as E812SYS.
This is especially helpful if one is running multiple instances of EnterpriseOne on one iSeries machine.
Important: When the data is far from the machine processing it, network latency due to database I/O is a
factor. Also, when the database server and the processing server are running different operating systems,
translation and middleware overhead between the two unlike systems is often a huge factor.
Slower performance between unlike systems is especially true if one of the machines is an iSeries. The integrated
nature of the DB2 database with the iSeries operating system gives iSeries a particular advantage; the DBMS is not
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 13
an application, it is actually part of the operating system. However, if an UBE is run on a Windows machine with
the data residing on iSeries, then this unique advantage is gone, and a large performance penalty will result. The
overhead of the ODBC layer and the QZDASOINIT processing will result in a much longer runtime in the
Windows + iSeries configuration than if both data and processing were on the iSeries.
Please see the diagrams below which depict the iSeries+iSeries and iSeries+Windows scenarios.
It is not recommended to:
• Map BSFNs or UBEs to a Windows server to save CPU cycles on a massive iSeries box.
• Use a large and expensive iSeries box exclusively as a database server.
Other considerations include:
• If the real concern is to reduce the CPU usage on the iSeries machine, then the “Logic on Windows + Data
on iSeries” approach might be a prudent one. However, realize that the UBE in question will run two or three
longer in this configuration.
• The size and CPU capacity of servers, as well as the speed of the link between machines, will determine the
level of degradation.
Important: Most software tends to perform better the fewer resources it requires from remote or different
systems. Configurable Network Computing (CNC) allows data and logic to be distributed, but tradeoffs still exist.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 14
instance
DB2
BSFN call
UBE jdenet_k
Call Object
Kernel
OCM
ODBC
Master Business Functions (MBFs) called from within most UBEs are large and can have literally thousands of
database I/O. In fact, most Oracle JD Edwards EnterpriseOne I/O occurs in C business functions.
If the BSFNs are not running on iSeries, then each I/O is a network roundtrip. This leads to a great deal of latency
due to “network chattiness”. There is also the ODBC and QZDASOINIT overhead.
If this BSFNs logic is run on iSeries:
• Thousands of database I/O calls now occur locally to the iSeries via DBDR (for OneWorld XE) or XDA
(Oracle JD Edwards EnterpriseOne 8.9 or higher).
• The XDA and DBDR calls happen via service program APIs called from the UBE job. No extra system jobs
are required, so there is no additional middleware overhead. Also, no ODBC layer and no QZDASOINIT
jobs are involved.
• Network latency is dramatically reduced.
• Memory consumption is dramatically reduced since QZDASOINT jobs are not used.
Note: A UBE runs three times slower in the "Logic on Windows + Data on iSeries" configuration compared to
the configuration where both data and logic reside on the iSeries.
Qaqqini
Some iSeries SQL configuration file settings impact the processing of UBEs such as:
• OPTIMIZATION_GOAL=*ALLIO
This setting works best for long UBEs processing large rowsets.
• OPTIMIZATION_GOAL=*FIRSTIO
This setting will slow down long UBEs, but works best for interactive queries.
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 15
UBE LOGGING AND DEBUGGING
Some suggestions to improve performance include changing logging settings.
• DO NOT use a UBE debug setting higher than ‘0’ when generating debug logs for performance analysis
using the Performance Workbench tools.
[UBE]
UBEDebugLevel=0
Values of ‘1’ or greater generate a large amount UBE-specific data without timestamps. This data is not
useful for performance, can skew the timings, and is disk space intensive. Much smaller logs will result if
this setting is greater than zero.
The UBEDebugLevel parameter can also be specified at runtime when launching UBEs via Batch
Versions, eliminating the need to change the server JDE.INI, which affects all other UBEs:
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 16
1. Check this box 2. Set this to ‘0’
Same as: Same as:
[DEBUG]
Output=FILE [UBE]
UBEDebugLevel=0
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 17
• A robust enough sample of the code is needed to determine what’s happening
• See the “Skewed Extrapolations” section.
Important: A debug log is only as good as the use case which is used to generate it.
• DO NOT run a server UBE on the fat client for the purpose of capturing logs. Get a server log.
• Many essential components of UBEs are disabled on the client and work only on the server.
This invalidates the UBE profile in the log. Some examples of functionality which does not work
on the client include the following:
• Buffered Inserts
• JDB Caching via API
• Subsystems
• The data will be across the network if the job is run on the client, leading to much more latency.
• Client side logs contain other threads of execution related to the interactive client which make
the debug log confusing and difficult to read.
• Starting with Oracle JD Edwards EnterpriseOne 8.11, the fat client is no longer a supported
environment for running applications other than the development tools.
• ALWAYS look at jde.log FIRST
• The jde.log is the error log, and is often the source of low-hanging fruit which will save a lot of
time by circumventing a lot of painstaking analysis.
• Example: large numbers of “Duplicate Key INSERT” errors or other SQL failures are
commonly found in failing or poor performing UBEs. This can lead directly to the discovery of
data or configuration problems
Skewed extrapolations
In particular with long running UBEs, the first few minutes of the run are spent in startup and initialization tasks
which are not repeated.
• There may be also a time-consuming data selection query at the start of the run that is not repeated.
• Throughput numbers will often not be accurate if extrapolated from only a few minutes.
• Data caching may also increase the throughput over a period of time.
• In long running UBEs, throughput for hour #2 is often greater than hour #1
It is not recommended to measure the throughput of a UBE until startup overhead is complete:
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 18
Begin End
Important: Use a large enough data sample when measuring UBE throughput
[DEBUG]
Output=FILE
;Output=NONE
©Copyright Oracle 2007. All rights reserved. UBE Performance and Tuning ¡ Page 19
Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.
Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
oracle.com
This document is for informational purposes only and may not be incorporated
into a contract or agreement.