Practice - Detecting CPU Bottlenecks P a g e |1

Practice 27

Detecting CPU Bottlenecks

Practice Target
In this practice, you will perform the following:
• Diagnose CPU-bound issue from external sources

• Diagnose CPU-bound issue from within the database

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |2

Diagnosing CPU-bound Issues

Scenario 1

In this section of the practice, you will simulate a CPU stress from external process and use AWR
reports to troubleshoot it.

1. In the hosting PC, open a command prompt window and start Swingbench.
D:
cd D:\swingbench\winbin
set PATH=D:\oracle\product\12.1.0\client_1\jdk\jre\bin;%PATH%
swingbench.bat

2. In Swingbench, open the oltp.xml configuration file then Start Benchmark button.

3. Start Putty and connect to srv1 as oracle, invoke SQL*Plus and login to ORADB as sysdba. In the
rest of this practice, this session will be referred to as admin window.
sqlplus / as sysdba
set sqlprompt "ADMIN> "

4. Open another Putty session. Connect to srv1 as root, then run the following command to
simulate a load on the CPU from external process for 5 minutes. This Putty window will be
referred to as the testing window in the remaining part of the practice.
The command applies a CPU stress on the machine for 5 minutes.
stress --cpu 20 --timeout 300s

5. In the admin window, run the following script to create two AWR snapshots in 5 minutes.
BEGIN
DBMS_SCHEDULER.CREATE_JOB (
JOB_NAME => 'CREATE_2_AWR_SNAPSHOTS',
JOB_TYPE => 'PLSQL_BLOCK',
JOB_ACTION => 'BEGIN DBMS_WORKLOAD_REPOSITORY.CREATE_SNAPSHOT(); END;',
START_DATE => SYSDATE,
REPEAT_INTERVAL => 'FREQ=MINUTELY;INTERVAL=5', /* every 5 minutes */
END_DATE => SYSDATE + (1/24/60*6),
AUTO_DROP => TRUE,
ENABLED => TRUE,
COMMENTS => 'create an AWR snapshots: now and after 5 minutes');
END;
/

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |3

6. Wait until 5 minutes is passed.

7. In the admin window, retrieve list of the most recent AWR snapshots and make sure two
snapshots have been created in the last 5 minutes.
@ $SD/awr_today_snapshots.sql

8. Produce an AWR report for the snapshots that you manually created earlier. Give the produced
report the name awr_ext_cpu.html

@ ?/rdbms/admin/awrrpt.sql

9. Move the produced file to the staging folder.

host mv awr_ext_cpu.html /media/sf_extdisk

10. In the hosting PC, open the produced report with your favorite browser. Observe the following in
the listed sections in the report:
• Report header:
- maximum CPU time for the report period is 2 (# of CPU) x 5 minutes = 600 seconds

• Top 10 Foreground Events by Total Wait Time:

- Observe the DB CPU-> Total Wait Time (can be obtained from Time Model as well)
- When comparing to the maximum CPU time, it is a small percentage. Till now, there is no
obvious issue.

• Wait Classes by Total Wait Time:

- Same information as the preceding section.

• Host CPU:
- Load average is greater than two times the number of CPUs. This is an indication that the
CPU could be a bottleneck.
- Total consumption from the CPU power = %User+%System+%WIO
- They represent high percentage of the 100%
- As the DB CPU was a low percentage, this means that the most CPU consumption was from
external process. But there is a slight possibility that the difference in CPU consumption was
taken from background process (although this is very rare). To confirm, we go to the Time
Model section.

• Time Model Statistics:

- 'background cpu time' is of low percentage. The process that was hammering the CPU is
some process other than the Oracle database.

Note: The '% Total CPU' is the 'total CPU time' (in Time Model section)/Maximum CPU time

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |4

Note: In Time Model section, do not be confused by the column '% of Total CPU Time'. That is
not the percentage of the DB CPU from the maximum CPU time. It is the percentage of the DB
CPU from the total database CP consumption. It is calculated by the following formula:

'% of Total CPU Time' = DB CPU/( DB CPU + background cpu time )

• Operating System Statistics Details

- %user is high

Question: Can we use ASH to detect this issue?

Note: Although we could detect CPU-bound issue caused by external source from AWR reports,
usually they are detected by monitoring the CPU consumption from OS level. We will cover monitoring
CPU form OS level later in the course.

Note: do not stop Swingbench sessions.

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |5

Scenario 2
In this section of the practice, you will simulate a CPU-bound situation from the database and use
AWR reports to troubleshoot it.

11. In the testing Putty window, change the current user to oracle, then run the following script to
kick off 4 SQL*Plus sessions that apply load on the CPU for 4 minutes. Do not wait for the script
to finish. Go to the next step.
su - oracle
$SD/apply_soe_cpu.sh 4 240

12. In the admin window, run the following code to create two AWR snapshots in 5 minutes.
BEGIN
DBMS_SCHEDULER.CREATE_JOB (
JOB_NAME => 'CREATE_2_AWR_SNAPSHOTS',
JOB_TYPE => 'PLSQL_BLOCK',
JOB_ACTION => 'BEGIN DBMS_WORKLOAD_REPOSITORY.CREATE_SNAPSHOT(); END;',
START_DATE => SYSDATE,
REPEAT_INTERVAL => 'FREQ=MINUTELY;INTERVAL=5', /* every 5 minutes */
END_DATE => SYSDATE + (1/24/60*6),
AUTO_DROP => TRUE,
ENABLED => TRUE,
COMMENTS => 'create an AWR snapshots: now and after 5 minutes');
END;
/

13. Wait until the 5 minutes is elapsed.

14. Make sure the AWR snapshots were created.

@ $SD/awr_today_snapshots.sql

15. Produce an AWR report for the create snapshots. Give the produced report the name
awr_db_cpu.html
@ ?/rdbms/admin/awrrpt.sql

16. Move the produced file to the staging folder.

host mv awr_db_cpu.html /media/sf_extdisk

17. In the hosting PC, open the produced report with your favorite browser.

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |6

18. In the hosting PC, open the produced report with your favorite browser. Observe the following in
the listed sections in the report:
• Report header:
- maximum CPU time for the report period is 2 (# of CPU) x 5 minutes = 600 seconds

• ADDM Findings:
- 'CPU Usage' is one of the ADDM findings.

• Top 10 Foreground Events by Total Wait Time:

- Observe the DB CPU-> Total Wait Time (can be obtained from Time Model as well)
- When comparing to the maximum CPU time, it is a high percentage. This means the
database was consuming much of the CPU power during the report time.

• Wait Classes by Total Wait Time:

- Same information as the preceding section
- If you compare the 'Avg Active Sessions' value to its value in scenario 1, it is much higher
in scenario 2. It is also close to the number of CPUs. These are clear indications that the
CPU consumption was within the database.

• Host CPU:
- Total consumption from the CPU power = %User+%System+%WIO
- They represent high percentage which means the CPU has heavily consumed

• Instance CPU:
- %Busy CPU is very high. This means that the most CPU consumption was from the
database side.
Note: “%Busy CPU” = Total Database CPU time / CPU Busy Time

• Time Model Statistics:

- No new information about the CPU consumption is obtained.
After we determine that some database sessions were taking much of the CPU time, let try to detect
the SQL statements that were consuming this time.

• SQL ordered by CPU Time:

- The culprit PL/SQL program unit is detected.

Question: Can we use ASH to detect this issue?

Cleanup

19. (optional) Close the browser window and delete the files:
host rm /media/sf_extdisk/awr_ext_cpu.html
host rm /media/sf_extdisk/awr_db_cpu.html

Oracle Database Performance Tuning, a course by Ahmed Baraka

 Practice - Detecting CPU Bottlenecks P a g e |7

Summary

• AWR reports can be used to detect CPU-bound issues from external sources and from within the
database.

• If a CPU issue is caused by external sources, AWR cannot help on detecting the process which
was responsible for hammering the CPU.

Oracle Database Performance Tuning, a course by Ahmed Baraka