BW Fine Tuning on Netweaver BW 730 Part 1 & 2

Standard DSO activation is always a performance bottleneck step among BW data loading
steps. In this blog, I will provide how DSO activation be improved on BW730 new features.
First let go through DSO activation logic to understand DSO activation flow.

3 DSO tables are involved for DSO activation process.

Active Table => /BIC/A####00
ZDSOTEST1)

(#### is DSO technical name, e.x.

Change Log Table =>/BIC/B<10 gen. digits>

Active Queue Table => /BIC/A####40

(#### is DSO technical name, e.x. ZDSOTEST1)

DSO activation job will use one batch job to fetch records from DSO activation queue table
(/BIC/####40) package by package. The DataPackage size is depended on the specified
DSO activation DataPacakge size defined in transaction RSODSO_SETTINGS. We can
call this job as main job. Once the main job get records from /BIC/A####40, it will trigger
several child jobs to write the records to active data table (/BIC/A####00) and change log
table (/BIC/B<10 gen. digits>). We call such writing job as child job.
In BW701 DSO activation is suffering performance bottleneck with single record lookup
from DSO active table. I use BW test environment for both BW701 and BW730 as below:
BW 701 vs BW730 test environment:
BW Test Environment

Support Package Level

Database

BW701

BW 7.0 EhP1 SP08

Oracle 11.2.0.1

BW730

Bw7.3 SP03

Oracle 11.2.0.2

Also used worst DSO activation scenario as below:

1. Load 1 million records to DSO object as delta request, which have no overlap with
previous delta init request.
2. Activate the single 1 million record delta request in both BW701 and BW730
environment with same DSO activation settings.
a. DataPackage Size: 50,000
b. Parallel Degree: 5
c. Run Mode: Batch Mode
d. No SID generation during activation
e. No unique record

f. Total volume: 1 million records

(* sorry for my poor testing environment, there are only10 background processes available)
Here is improvement on DSO activation in BW730 framework with same 1 million records
volume.

In general DSO activation job performance improved 43% in BW730 test environment.
DSO activation runtime reduced from 309 seconds to 176 seconds. The main difference is
on each single child writing job runtime. Each single child job average runtime reduced
from 49 seconds to 22 seconds.
ABAP trace of single child job is captured for both BW701 and BW730 running. First let see
the famous mass single select in BW701 child job.

74% of child runtime is spent on 50000 single select lookup calling on DSO active table.
Then let see how DSO activation child job works in BW730.

50,000 single select disappear!!! Now SAP developers replace 50000 single select with
only one select lookup calling on DSO active table. The main change on BW730 DSO
activation is that select singleABAP codes are replaced with select.for all entries as I
captured in generated DSO program.

Above ABAP code change reduced DSO active table lookup runtime reduce from 37
seconds to 10 seconds. In sequence the whole DSO activation job runtime benefit from this
major logic changes in BW730.
I was told in SAP SDN blog Performance Improvements for DataStore Objects that DSO
activation job can get 10% ~ 30% improvement in BW730.
Performance Improvements for DataStore Objects
My small comparison is based on single DSO activation job scenario. With more DSO
object activation job running in parallel on real SAP customer environments, the DSO
activation performance would have certain decrease. But without any DSO setting
changes, DSO activation should be able to decreases by 10-30% for sure.
However this is not the end of tuning story for BW730 DSO activation job. In part 2 I will
provide more setting change tips based on my additional DSO activation tests in BW730
environment.

BW Fine Tuning on Netweaver BW 730 Part 2

In BW Fine Tuning on Netweaver BW 730 Part 1 I provide the DSO activation
performance improvement in BW730 based on my testing on both BW701 and BW730
environment.
In Part2 I will continue provide some DSO activation fine tuning tips based on the DSO
activation testing on pure BW730 environment.
The test scenario is same as below:
1. Load 1 million records to DSO object as delta request, which have no overlap with
previous delta init request.
2. Activate the single 1 million record delta request in both BW701 and BW730
environment with same DSO activation settings.
a. DataPackage Size: 50,000
b. Parallel Degree: 5
c. Run Mode: Batch Mode
d. No SID generation during activation
e. No unique record
f. Total Volume: 1 million records
However in my next test approaches, DataPackage size and activation parallel degree is
changed to compare the DSO activation runtime for different settings in BW730. Below is
the test result for different DSO activation DataPackage size and parallel degree.
Test round

DataPacakge Size

ParallelDegree

20000

20000

50000

50000

Below is my DSO activation tests result

On first impress for the test result, I am quite surprised on DSO activation job performance
regarding large DataPackage. With same parallel degree 5, test round 3 DSO activation job
took 176 seconds for 50,000 DataPackage size. However for 20,000 Datapacakge size and
same 5 parallel degree, DSO activation only used 160 seconds.

The main difference for test round 1 & 3 is on how DSO Active Queue extract main job run
with writing child jobs. Take ABAP trace for detail explanation:
Test Round 1: DataPackage Size 50000, Parallel Degree 5
Child job ABAP trace

Select FOR ALL ENTRIES Statement took 10 seconds to process 50000 records. Array
insert into change log and active tables took 8 seconds.
Test Round 3: DataPacakge Size 20000, Parallel Degree 5
Child Job ABAP trace

SelectFOR ALL ENTRIES statement took 4.6 seconds to process 50000 record lookup.
Array insert into change log and active table took 4.4 seconds.
From child job performance trace result, the child job SQL statement runtime is linearly
increased with increased record number. The main difference for this 2 test running is on
the behavior of Active queue main job.
Test Round 1 Main Job ABAP Trace

Test Round 3 Main Job ABAP Trace

Active queue table (/BIC/AZDSOTEST40) fetch throughput is similar for both tests running.
The only difference is on the difference that main job waiting for child writing job to be
released for next writing task. In ABAP trace, function CALL ALERTS runtime reflect this
waiting difference for both test running.
With 20000 DataPackage size, main job spent 95 seconds on waiting free child jobs. With
50000 DataPackage size, main job spent 113 seconds on waiting free child jobs. Because
for both tests running, parallel degree is only set with 5, which means at most 4 child jobs
can be initiated for writing tasks.
On test round 2 & 4, when parallel degree increased to 8, more background work process
can be used for child writing job, the DSO activation runtime reduced mainly on 50% less
main job waiting time.
Based on above tests running, the DSO activation tuning approach should follow below
sequence:
1. Make sure normal tuning approaches have been applied. For example uncheck
generate SID during DSO activation.
2. Increase parallel degree as first priority to reduce DSO activation main job waiting
time.
3. If parallel degree increase still cannot improve DSO activation as expected, then
reduce DataPackage size to let DSO activation child writing job take less runtime.
In addition the DSO activation main job waiting time will also be reduced on waiting
time slots.
Finally compared with BW701 same data volume DSO activation, the final DSO activation
jobreduced from 309 seconds to 139 seconds (55% improvement) with below changes:
1. Utilize of SAP BW730 mass lookup for DSO active table. (43% improvement)
2. Reduce DataPackage size from 50000 to 20000. (2% improvement)
3. Increase parallel degree from 5 to 8. (10% improvement)
On large scale BW system with huge DSO data volume and more ABAP instances, DSO
activation job performance can gain much more improvement with BW730 DSO new
features.

At last you can also refer below SAP Notes to improve DSO activation job on DB6
database:

SAP Note 1515687: DB6: Performance optimization DSO data activation

SAP Note 1421011: DB6: Speed up selection from DSO activation queue (DB2
V9.1)

In general please follow below SAP Notes to calculate Datapacakge size and parallel
degree:

SAP Note 1567541P27:DSO: Poor performance when you start request activation
SAP Note 1392715: DSO req. activation:collective perf. problem note
SAP Note 1599602: FAQ: BW system performance