Oracle 23c 02

Oracle Database
Photo by JESHOOTS.COM on Unsplash

Release and Patching Strategy
Virtual Classroom Series – Episode 16
1 Copyright © 2023, Oracle and/or its affiliates

ROY SWONGER
Vice President
Database Upgrade, Utilities & Patching
royfswonger
@royfswonger

MIKE DIETRICH
Senior Director Product Management
Database Upgrade, Migrations & Patching
mikedietrich
@mikedietrichde
https://mikedietrichde.com

DANIEL OVERBY HANSEN
Senior Principal Product Manager
Cloud Migrations
dohdatabase
@dohdatabase
https://dohdatabase.com

RODRIGO JORGE
Senior Principal Product Manager
Database Patching and Upgrade
rodrigoaraujorge
@rodrigojorgedba
https://dbarj.com.br/en

Webinar | Get The Slides
https://MikeDietrichDE.com/slides

Recorded Web Seminars
https://MikeDietrichDE.com/video
s
More than 30 hours of technical content,

on-demand, anytime, anywhere

release and Release Strategy
Oracle Database Installation and Patches

patching strategy Upgrade and Patching
Patching Basics
Patching Methods
Grid Infrastructure Patching
Quarterly Patching
Patching Insights
Summary

Lifetime Support Policy
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
MARKET
Oracle 11.2 WAIVED EXTENDED EXTENDED DRIVEN
MARKE
Oracle 12.1 EXTENDED T
DRIVEN
LIMITE
Oracle 12.2.0.1 D
Oracle 18
(12.2.0.2)
Oracle 19 EXTENDED
(12.2.0.3)
Oracle 21
Premier Support Waived Extended Support Paid Extended Support Market Driven Support Limited Error Correction

Release Strategy | Make Your Plan
We are here
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
MARKET
11.2.0.4 DRIVEN
12.1.0.2 MDS
12.2.0.1
18c
Oracle 19
(12.2.0.3) EXTENDED
Oracle 21
EXTENDED
Oracle Database 23c Support

Release Types
LONG TERM INNOVATION

SUPPORT 2 years of Premier Support
5+ years of Premier Support No Extended Support
followed by
3+ years of Extended Support
Innovation
Long Term Support Extended
Innovation
11 Copyright © 2023, Oracle and/or its affiliates Long Term Support Extended
Move production databases from one
Long Term Support release to the

next

Next Long Term Support release
Oracle Database
23c
Upgrade possible only from:
• Oracle Database 19c

• Oracle Database 21c

Do you want to upgrade?
Oracle Database
11.2.0.4
Oracle Database
12.1.0.2
Oracle Database
12.2.0.1
Oracle Database 18c

Oracle Database 11.2.0.4
Oracle Database 12.1.0.2 Oracle Database
Oracle Database 19c
Oracle Database 12.2.0.1 23c
Oracle Database 18c

Direct upgrade is NOTOracle
Oracle
Database
Database
Oracle Database 23c
11.2.0.4
12.1.0.2
Oracle Database 12.2.0.1
possible!! Oracle Database 18c

Everybody must upgrade to Oracle
Database 19c, with or without
Multitenant

3
user-created
pluggable databases
are always included - even in SE2
since Oracle Database 19c
Have at least a few environments in
Oracle Database 19c using Oracle
Multitenant


Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

Your path to successful database upgrades / migrations
1 2 3
Install Oracle Home Download and deploy Collect performance
including RU and MRP the most recent information from
AutoUpgrade current source and
MOS Note: 2118136.2 test thoroughly
MOS Note: 555.1 MOS Note: 2485457.1
MOS Note: 2781612.2

You always start with
Oracle Database 19c base release
• Oracle Database 19.3.0

Before upgrading, apply the most recent RU
• Use the Patch Download Assistant MOS Note: 2118136.2

Release Update Contents
1200 1140
1000 971
874
800
721 718
654
603
600 573
533
502
472 474
398
400 334
294
228
200
0
19.3.0 19.4.0 19.5.0 19.6.0 19.7.0 19.8.0 19.9.0 19.10.0 19.11.0 19.12.0 19.13.0 19.14.0 19.15.0 19.16.0 19.17.0 19.18.0 19.19.0
Database 19 Release Updates and Revisions Bugs Fixed Lists (Doc ID 2523220.1)

You are missing out if you don't apply
a recent Release Update

• Thousands of fixes
• 243 security fixes

Before upgrading, apply the most important patches
• Always use Important Recommended One-Off Patches: MOS Note: 555.1

Monthly Recommended Patches
A collection of recommended one-off fixes

provided at monthly intervals
via a single downloadable patch

Timeline | Release Updates
2021 2022 2023 2024

January April July October January April July October January April July October January April July
19c 19.10.0 19.11.0 19.12.0 19.13.0 19.14.0 19.15.0 19.16.0 19.17.0 19.18.0 19.19.0 19.20.0 19.21.0 19.22.0 19.23.0 19.24.0
21c 21.3.0 21.4.0 21.5.0 21.6.0 21.7.0 21.8.0 21.9.0 21.10.0 21.11.0 21.12.0 21.13.0 21.14.0 21.15.0

Monthly Recommended Patches | Timeline
2022 2023
October November December January February March April May June July August September October November
19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4 MRP5 MRP6
19.20.0 19.20.0 MRP1 MRP2 MRP3 MRP4
19.21.0 19.21.0 MRP1

Monthly Recommended Patches | Step-by-Step
2022 2023
19.17.0 19.17.0 MRP1

2022 2023
19.17.0 19.17.0 MRP1 MRP2

2022 2023
19.17.0 19.17.0 MRP1 MRP2 MRP3
19.18.0 19.18.0

2022 2023
19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4
19.18.0 19.18.0 MRP1

2022 2023
19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4 MRP5
19.18.0 19.18.0 MRP1 MRP2

2022 2023
19.18.0 19.18.0 MRP1 MRP2 MRP3
19.19.0 19.19.0

2022 2023
19.18.0 19.18.0 MRP1 MRP2 MRP3 MRP4
19.19.0 19.19.0 MRP1

2022 2023
19.20.0 19.20.0 MRP1 MRP2 MRP3 MRP4
19.21.0 19.21.0 MRP1

MRP Facts | Contents
Monthly Recommended Patches get sourced mostly from MOS Note 555.1

An MRP is a collection of
several one-off patches
• Delivered via a merge patch

An MRP does not change
the release number
• Like v$instance.version_full

MRPs are cumulative
but only within one MRP line
• Example: 19.17.0 MRP5 contains all previous MRPs
done for Oracle 19.17.0

MRPs may contain security fixes
• Release Updates remain the primary

security fix delivery mechanism

Soon, MRPs may be available in OCI
• Including ExaDB-C@C

MRP Facts | More Information
• Introducing Monthly Recommended Patches (MRPs) and FAQ (Doc ID 2898740.1

)
• Patching News: RURs are gone – long live MRPs (Blog Post)
• Oracle Database 19c Important Recommended One-off Patches (Doc ID 555.1)
• Oracle Database Patch Maintenance

Before upgrading, apply the most important patches
• In addition, use Patches to consider for 19c: MOS Note: 2781612.2

Apply the Data Pump Bundle Patch
• Data Pump Recommended Proactive Patches

For 19.10 and Above (Doc ID 2819284.1)

Data Pump Bundle Patch
Fewer Bugs
Important patches are included.
Monitor for bugs that affects many customers.
Faster Patching
The bundle patch changes the way Data Pump
is patched. Subsequent patches apply faster.

Data Pump Bundle Patch - MOS Note: 2819284.1
125
fixes
Data Pump Bundle Patch for
19.19.0

Why aren't these fixes included in an
RU?

Data Pump Bundle Patch is not
RAC Rolling and Standby-First
Installable

But ... it's much easier than it looks
like

Data Pump Bundle Patch Contents
Bundle Patch contains only:

• sql
• plsql
• xml
But it does not contain any files

which require a compilation/make of
rdbms
 It can be applied ONLINE

OPatch continues with these patches: 34620690
Do you want to proceed? [y|n]

y
User Responded with: Y
All checks passed.
Backing up files...
Applying interim patch '34620690' to OH
'/u01/app/oracle/product/19'
Patching component oracle.rdbms, 19.0.0.0.0...
Patching component oracle.rdbms.dbscripts, 19.0.0.0.0...

Patch 34620690 successfully applied.

When you run datapatch, ensure
that there are no active Data
Pump jobs

Non-Binary Online Patching Safeguards
Installing the Data Pump Bundle Patch when Data Pump is in use:
• Built-in 3-minute timeout before signaling an error
BEGIN ku$_dpload.initial_phase; END;

*
ERROR at line 1:
ORA-20000: Retry dpload.sql script later when
Data Pump and Metadata API are not in use; current users are:
pid:11720, user:SYS, machine:<Machine>, sid:263,
module:sqlplus@<ConnectString> (TNS V1-
ORA-06512: at "SYS.KU$_DPLOAD", line 1042
ORA-06512: at line 1

Attempting to run Data Pump while patching is in progress:

Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
ORA-31626: job does not exist
ORA-31637: cannot create job SYS_EXPORT_FULL_01 for user SYSTEM
ORA-06512: at "SYS.KUPV$FT", line 1142
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-39062: error creating master process DM00
ORA-39107: Master process DM00 violated startup protocol. Master error:
…
Note:
With the 19.14 (or later) Data Pump Bundle Patch installed you will see a much better error
message:
• ORA-39442: Data Pump software update in progress
Since Data Pump Bundle Patch 19.14.0 you'll receive:

Before, you would see:

ORA-31637: cannot create job SYS_EXPORT_FULL_01 for user SYSTEM
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-39062: error creating master process DM00
ORA-39107: Master process DM00 violated startup protocol. Master error:
…

Once applied, Data Pump Bundle
Patch speeds up future patching
significantly

Importing a complete application with
data goes from almost 2,5 hours to 48
minutes
– by just applying the Data Pump Bundle
Patch
Global provider of financial services


Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

AutoUpgrade automates your
migration
to Multitenant completely
Including Transparent Data Encryption
with AutoUpgrade's new keystore functionality

AutoUpgrade | Overview
11.2.0.4 1. 2. 3.
12.2.0.1
12.1.0.2
18c
12.2.0.1
18c ONE BUTTON APPROACH 19c
21c
19c But giving you all the control you want
CONFIG FILE ANALYZE UPGRADE
Optionally,
convert to PDB
63 Copyright © 2023 Oracle and/or its affiliates

Always use the latest
version of
AutoUpgrade
Download from My Oracle Support
(2485457.1)

$ java -jar autoupgrade.jar -version
build.version 23.1.230224
build.date 2023/02/24 14:53:24 -0500
build.hash a1e2990e
build.hash_date 2023/02/24 14:44:39 -0500
build.supported_target_versions 12.2,18,19,21
build.type production
build.label (HEAD, tag: v23.1, origin/stable_devel, stable_devel)

We made upgrading easy.
Now we make patching just as
easy.
AutoUpgrade functionality extended to patching

Patching
1
Install Oracle Home
2 3
Create a simple Start AutoUpgrade
including Release Update configuration file in deploy mode
and additional patches
(MOS Doc ID 555.1)

$ cat DB19.cfg
patch1.source_home=/u01/app/oracle/product/19.0.0.0/dbhome_19_18_0
patch1.target_home=/u01/app/oracle/product/19.0.0.0/dbhome_19_19_0
patch1.sid=DB19
$ java -jar autoupgrade.jar -config DB19.cfg -mode deploy

Patching
USE ANALYZE PROTECT AUTOMATE

Familiar interface Prechecks Resumable srvctl
Console Summary report Restoration /etc/oratab
Logging Restore point Files
Fallback Datapatch

Encryption
Hot clone
Refreshable clone
Patching RAC
Proactive fixups
Distributed upgrade
...
Patching
Out-of-place patching
support only right now
Further enhancements coming
soon

Patching
What's missing
Windows
RAC rolling
Data Guard standby-first

Patching
AutoUpgrade
Automate your patching process and
benefit from the familiar
AutoUpgrade
Fleet Patching and Provisioning

Go fleet scale with FPP and benefit from
additional functionality like deployment of
Oracle Home


Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

Basic Facts | What Is In A Patch
New or changed executables, libs or files
FILES bin/oracle
bin/srvctl
oracore/zoneinfo/timezone_34.dat
Apply and rollback scripts

sqlpatch/.../nnn_apply.sql
sqlpatch/.../nnn_rollback.sql
New or changed objects

SQL alter table sys.tab$ ...
create index sys.i_tab1 ...
PL/SQL create or replace package sys.dbms_scheduler ...

Basic Facts | How To Apply A Patch
$ORACLE_HOME/OPatch

Basic Facts | How To Apply A Patch
Applies binaries to an
opatch Oracle Home
All instances are down
Applies SQL and PL/SQL

datapatch changes to a database
Database is up

Basic Facts | What's Installed
What is installed in my Oracle Home?

OPatch
$ opatch lsinventory
$ opatch lspatches
DBMS_QOPATCH
SQL> select xmltransform(dbms_qopatch.get_opatch_lsinventory, dbms_qopatch.get_opatch_xslt) from dual;
Oracle Database 12.1 : FAQ on Queryable Patch Inventory (Doc ID 1530108.1)
What is installed in my database?

SQL> select * from dba_registry_sqlpatch;
SQL> select * from cdb_registry_sqlpatch;


Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

Think of
Patching
as like
crossing
this street
Photo by sq lim on Unsplash

Patching Concepts | In-Place Patching
Oracle Home, 19.18.0

19.19.0
STARTUP
SQL> SHUTDOWN
[oracle]$
[oracle]$ $ORACLE_HOME/OPatch/opatch
IMMEDIATE
$ORACLE_HOME/OPatch/datapatch apply
-verbose

Patching Concepts | Out-of-Place Patching
Oracle Home, 19.18.0
SQL> SHUTDOWN IMMEDIATE
[oracle]$
SQL> STARTUP
[oracle]$ $ORACLE_HOME/OPatch/opatch-verbose
$ORACLE_HOME/OPatch/datapatch apply
New or Cloned
Oracle Home
Home, 19.19.0
Twitter

Always patch out-of-place

Safely test and verify patches with
Standby-First Patch Apply
• Oracle Patch Assurance - Data Guard Standby-First Patc
h Apply (Doc ID 1265700.1)

Patching Concepts | Standby-First Patch Apply
Snapshot
standby
Oracle Home Redo Transport
19.18.0
RReed
dooT
Trraann
ssppoo
rrtt
New or Home
Oracle cloned Redo Transport
Oracle
19.19.0Home Primary
Standby
[oracle]$
DGMGRL>
[oracle]$convert database
switchover snapshot
$ORACLE_HOME/OPatch/opatch
$ORACLE_HOME/OPatch/datapatch
to PROD2; apply
PROD2 to physical standby;
-verbose

The patch must be marked as
Standby-First Installable
• Check the patch readme

Execute Datapatch on primary
database
• Execute as quickly as possible when all databases
are running on the new Oracle Home

Additional restrictions apply
• For details, check

Oracle Patch Assurance - Data Guard Standby-First Patc
h Apply (Doc ID 1265700.1)

Avoid downtime with
RAC Rolling Patching

Patching Concepts | RAC Rolling
User
Connections
Instances
Oracle Homes $ $ORACLE_HOME/OPatch/opatchauto
Nodes
• Clone Oracle Home
• Patch Oracle Home
• Move to new Oracle Home
• Execute datapatch
Database

RAC Rolling Patching | Best Practices
•If draining is a problem, patch

• Node by node
• In node groups
•Patch the standby cluster first

• Combine Standby First and RAC Rolling for premium protection
• Allows you to test on the standby cluster (functionality and rollback)

•Execute datapatch immediately after the last node has been

patched
•Execute datapatch on one node only

•Keep GI and DB patch level in sync

• This is what we test
• This is how we run our cloud
•Supported, but not recommended, combinations

• Grid Infrastructure home = 19.14.0
• Database home = 19.16.0
• Grid Infrastructure home = 19.18.0

• Database home = 19.14.0
• Node 1 – Grid Infrastructure home = 19.14.0

• Node 2 – Grid Infrastructure home = 19.17.0
• Patching node 1 on Monday, then patching node 2 on another day
• Operating a cluster with different Oracle versions of either Grid Infrastructure and/or Database homes on each of the nodes

Can I delay or omit applying patches to a
subset of instances/nodes in an Oracle RAC
No. All patching operations should be completed on all Oracle Real
cluster?
Application Clusters (RAC) instances as quickly as possible. When
applying patches to an Oracle Grid Infrastructure or Oracle
Database home, these patches must be made effective as soon as
possible, ideally within 24 hours. Rolling patching of RAC clusters
with the draining of connections is a recommended and well-tested
process. However, running mixed patch levels inherently brings
more risk, and testing any given patches in a mixed manner will be
less robust than trying a uniform set of patches across the cluster.
Running with a mix of patches across a cluster for an extended
time increases the risk of exposure to untested corner cases
impacting system stability, scalability, and potential availability.
Additionally, some functionality is restricted during rolling
patching.
RAC: Frequently Asked Questions (RAC FAQ) (Doc ID 220970.1)

•Potential risk when patching with delay

• Some cluster features may operate with limitations
• Some ASM operations may be limited or not possible
•Oracle strongly recommends patching as quickly as

possible
•Future versions of Oracle Grid Infrastructure may lift some

limitations

Release updates are always:
Standby-First installable
RAC Rolling installable

Patch
as quickly
Photo by Colby Winfield on Unsplash

as possible

Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

The following patching concepts
apply to Oracle Database patching as
well

0
Grid Infrastructure | Patching Concepts
ALL NODE
• All nodes patched at one time
• One long database outage
• Works for all patches, including non-
rolling
• Cluster at full capacity except for
outage
Databa
se outa
ge
Rolling Patch - OPatch Support for RAC (Doc ID 244241.1)

MINIMUM DOWNTIME
• Nodes patched in two batches
• One short database outage
• Works for all patches, including non-
rolling
• Other nodes must handle workload
while another batch is patched
Databa
se outa
ge

ROLLING
• Each node patched separately
• No database outage
• Patch must be RAC rolling installable
while one node is patched
No outa
ge

ROLLING IN GROUPS
• Patch a subset together
• Useful when draining is a problem
• No database outage
• Patch must be RAC rolling installable
while one node is patched
No outa
ge

Rolling installation of patches is
preferred, but requires effective
draining
• Optionally, consider a batched approach

5
Complete a rolling patch installation
as quickly as possible
• Ideally within 24 hours
• Certain functionality during the rolling patch installation

Grid Infrastructure | Draining
DRAINING
FAN event
• Allows users to finish their work
and reconnect to another instance
• New sessions connect to other
Instan instances
ce
going do • Sessions that don't drain in time
wn
are forcefully terminated
• Controlled by drain_timeout
parameter in srvctl and
DBMS_SERVICE

Setting drain_timeout very low
may cause a login storm
• Exercise caution on databases with many connections

Setting drain_timeout very high
may be risky
• Load is spread on fewer instances
• Cluster is in ROLLING PATCH state for an extended period

Comply with
Maximum Availability Architecture
• Continuous Availability - MAA Checklist for Applications
for the Oracle Database

Grid Infrastructure | Data Guard
ALTERNATIVE
• If draining is a problem
• Downtime limited to a switchover
• Test your Data Guard configuration
Switcho
SPtrainmdabryy ver Standby
Primary

Data Guard | Additional Information
INTERVIEW WITH LUDOVICO CALDERA

Ludovico is Data Guard Product Manager and
he shares his top tips for patching Oracle Grid Infrastructure and Data Guard
PATCHING ORACLE GRID INFRASTRUCTURE AND ORACLE DATA GUARD

Blog post with additional details plus instructions on
how to patch GI and database at the same time when you have Data Guard

Patching Methods
In-place
1 Replaces existing Oracle Home
Uses opatchauto
Out-of-place
2 Creates a new Oracle Home
Uses opatchauto or gridSetup

Use out-of-place patching
• Minimize downtime
• Minimize risk during outage
• Easier rollback

Patching Grid Infrastructure and Database
Option 1 Option 2
TOGETHER SEPARATELY
One maintenance window Two maintenance windows
Longer window Shorter, but more windows
Several changes One change at a time
Keep maintenance windows close to

each other

top tips
PATCHING SUCCESS
Cluster Verification Utility Use CVU before and after patching
Preferably through EXAchk or ORAchk

Patch Level
Identifies potential issues
Application Continuity Light-weight, non-intrusive
Always use the latest version

OPatch

top tips
PATCHING SUCCESS
Cluster Verification Utility Apply patches regularly
Apply recent Release Updates

Patch Level
Apply MRPs
Application Continuity Keep GI and DB patch levels in sync
OPatch

top tips
PATCHING SUCCESS
Cluster Verification Utility
Patch Level Completely

hide interruptions from users
Application Continuity Hides planned and unplanned events
Comply with MAA guidelines

OPatch
See also
Transparent Application Continuity

top tips
PATCHING SUCCESS
Cluster Verification Utility
Patch Level
Application Continuity
Always use the latest version of OPatch
OPatch Use in GI and DB homes

https://www.oracle.com/database/technologies/rac/fpp.html

0

Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

The Database Patching Process
FIND INSTALL APPLY

Find the right Download and Apply the
bundle and install the patches to the
one-off patches patches database

My database is
Photo by Ben White on Unsplash

not facing the internet

92 of malware gets
delivered via email
Photo by Clint Patterson on Unsplash

Finding the Right Patches | Quarterly
https://www.oracle.com/security-alerts/
Do I need to apply
this bundle?
How to evaluate the risk
Photo by Lubo Minar on Unsplash

Critical Patch Alert July 2022 | Risk Matrix
Oracle Java Virtual Machine
Do you have OJVM installed?
Photo by Michiel Leunens on Unsplash

Do you need OJVM?
Do you use OJVM?

Oracle Java Virtual Machine | OJVM
• OJVM Security Fixes – average risk score in quarterly patches: ⌀ 7.2

Highest single OJVM Base Score per Security Alert since January 2015
9.9 9.8
10.0
9.0 9.0 9.0 9.0 9.0 9.0 9.1 9.0

9.0
8.5
8.2 8.3
8.0 8.0
8.0
7.5 7.5
7.1
7.0 6.8 6.8 6.8
6.5 6.5 6.5
6.0
5.3 5.3 5.3
5.0 4.8
4.3 4.3 4.3 4.3
4.0
3.5
3.1
3.0
2.0
1.0
0.0
5 5 5 5 6 6 6 6 7 7 7 7 8 8 8 8 9 9 9 9 0 0 0 0 1 1 1 1 2 2 2 2 3 3
n -1 r-1
u l-1 t -1 n -1 r-1
u l-1 t -1 n -1 r-1
u l-1 t -1 n -1 r-1
u l-1 t -1 n -1 r-1
u l-1 t -1 n -2 r-2
u l-2 t -2 n -2 r-2
u l-2 t -2 n -2 r-2
u l-2 t -2 n -2 r-2
c c c c c c c c
Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap J O Ja Ap
Check | OJVM
• Is OJVM installed?
select comp_id, comp_name, version from dba_registry order by 1;

COMP_ID COMP_NAME VERSION
CATALOG Oracle Database Catalog Views 19.19.0.0.0

CATJAVA Oracle Database Java Packages 19.19.0.0.0
CATPROC Oracle Database Packages and Types 19.19.0.0.0
JAVAVM JServer JAVA Virtual Machine 19.19.0.0.0
OLS Oracle Label Security 19.19.0.0.0
ORDIM Oracle Multimedia 19.19.0.0.0
OWM Oracle Workspace Manager 19.19.0.0.0
XDB Oracle XML Database 19.19.0.0.0
XML Oracle XDK 19.19.0.0.0

OJVM Patching | Option 1
Patch PATCH QUARTERLY

Disable
Remove • OJVM bundle patch is a separate download until Oracle
19c
• From Oracle 21c onward, OJVM is part of the RU

Patch Single instance

Disable • Database is down
Remove
Real Application Cluster

• Database stays up
• Each instance must go down in a rolling manner
• See MOS Doc ID 2217053.1 for details

Patch Oracle Database 21c / Fully RAC Rolling installable

Disable 23c No interruption during
datapatch
Remove
RAC Rolling installable
Oracle Database 19c No datapatch downtime but:
• Java subsystem is patched which
requires ~10 second outage
• Connected clients using OJVM
will receive ORA-29548

Patch DISABLE OJVM

Disable using mitigation patch
Remove
• Disables the Java subsystem
• Java subsystem must be re-enabled during patching and
upgrade
• Can be used on PDB level

Patch Mitigation patch present in all Oracle Homes since 2018

Disable
Remove Further Information
• MOS Note: 1929745.1
• OJVM and the Mitigation Patch – Things to Know
• Do you need the Mitigation Patch in CDB$ROOT and all PDBs?

Patch How do you use the Mitigation Patch?

Disable
• Enable patch @?/rdbms/admin/dbmsjdev.sql
Remove
• Disable OJVM exec DBMS_JAVA_DEV.DISABLE
• Re-enable OJVM, exec DBMS_JAVA_DEV.ENABLE

e.g. before database patching
• Important note:
Disable OJVM with the Mitigation Patch in PDB$SEED to prevent new PDBs
being provisioned with an enabled OJVM – see Blog Post
Patch REMOVE OJVM

Disable
Remove
Exercise caution in an existing database
Conduct thorough testing
Non-CDB MOS Note: 2314363.1

CDB MOS Note: 2262919.1
See also JAVAVM and XML Cleanup in the database

No STARTUP UPGRADE for datapatch
Not even when you patch OJVM
• Even if the readme says so
• See blog post for details
• If needed, use ./datapatch -skip_upgrade_check


Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

FIND INSTALL APPLY


Download & Install Patches
Check for newest opatch version

Download and unzip patch
(bundle)
Check for patch conflicts
Shutdown instance using this
home
Apply binary with opatch or
opatchauto

Always use the latest OPatch

• Download Patch 6880880
• MOS Note: 274526.1
• Why should you use the most recent version of OPATCH?

If OPatch takes too long, use out-of-
place patching with a brand-new
Oracle Home
• Avoid cloned Oracle Homes and in-place patching

Reusing Oracle Home leads to:
• Disk footprint increases
• Patching runtime increases

Patching Performance
Root cause for long opatch runtime

• opatch maintains an ever-growing patch history in
$ORACLE_HOME/inventory/ContentsXML/oui-patch.xml
• This file gets queried multiple times

and can delay patching massively
Solution
• Clean up available since opatch 37
opatch util deleteinactivepatches

Patching Performance | Clean up
Original oui-patch.xml with 12 RUs Cleaned oui-patch.xml with only 2 RUs
• 1001 lines • 136 lines
• Space check: 5:29 min • Space check: 0:09 min

• Conflict check: 5:26 min • Conflict check: 0:08 min
• Check during apply: 5:29 min • Check during apply: 0:12 min
• Raw patch apply: 3:58 min • Raw patch apply: 3:51 min
• Total time taken: 19:22 min • Total time taken: 4:20 min

Unzip new Oracle Home
1 Download base release from oracle.com
Update OPatch
2 Download from Oracle Support
Install Oracle Home and patches

3 Attach Oracle Home and apply patches in one operation
./runInstaller –applyRU ... –applyOneOffs ...
Check blog post for details

FIND INSTALL APPLY


Patching a Database
1 2
Start database in new Oracle Complete patching with
Home datapatch
Start in normal open Found in $ORACLE_HOME/OPatch

Open all PDBs One database per invocation
Multiple datapatch sessions in parallel
Datapatch User Guide (Doc ID 268052
1.1)
Recompile invalid objects
before invoking datapatch

Patching Timeline
utlrp
shutdown immediate

The database must be open
Only open PDBs are patched
• Upgrade mode or restricted session is not needed

Patching Timeline
utlrp
shutdown immediate
startup [open]
alter pluggable database all open

You can run datapatch while users
are connected to the database
• Details in blog post

Patching Timeline
Users can connect again Patching completed
Potential concurrency
utlrp
shutdown immediate datapatch
startup [open]
Recompilation

Concurrency
• Datapatch waits 15 min to acquire a lock

• On timeout, ORA-04021 timeout occurred while waiting to lock object
• Optionally, find blocking session and kill it

• How to Analyze Library Cache Timeout with Associated: ORA-04021 'timeout occurred
while waiting to lock object %
s%s%s%s%s.' Errors (Doc ID 1486712.1)
• Timeout period controlled by _kgl_time_to_wait_for_locks

• Use under guidance of Oracle Support

Datapatch uses REGISTRY$SQLPATCH to
control the patching operations

Recompilation
• Datapatch recompiles objects invalidated during patching
• If more than 300 objects are invalidated no recompilation takes

places
• Recompile manually
• Or, objects will be recompiled on usage
• Adjust the threshold

datapatch ... -recomp_threshold 300
• Consider recompiling invalid objects after patching

Patching Timeline
Recompile manually
if more than 300 objects
utlrp
shutdown immediate datapatch utlrp
startup [open]
Recompilation only
if less than 300 invalid objects

$ pwd
/u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_485_2022_01_19_22_13_40
$ grep "recomp_threshold" *catcon* -A1

[CDB$ROOT] Invalid ORACLE_MAINTAINED objects: before patching=0, after patching=0, recomp_threshold=300
[CDB$ROOT] All ORACLE_MAINTAINED objects are VALID, recompilation not needed.
[PDB$SEED] Invalid ORACLE_MAINTAINED objects: before patching=0, after patching=0, recomp_threshold=300
[PDB$SEED] All ORACLE_MAINTAINED objects are VALID, recompilation not needed.

$ pwd
/u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_485_2022_01_19_22_13_40
$ grep "recomp_threshold" *catcon* -A1

[CDB$ROOT] Invalid ORACLE_MAINTAINED objects: before patching=0, after patching=0, recomp_threshold=300
[CDB$ROOT] All ORACLE_MAINTAINED objects are VALID, recompilation not needed.
[PDB$SEED] Invalid ORACLE_MAINTAINED objects: before patching=0, after patching=0, recomp_threshold=300
[PDB$SEED] All ORACLE_MAINTAINED objects are VALID, recompilation not needed.

$ORACLE_HOME/OPatch/datapatch
$ORACLE_HOME/sqlpatch/sqlpatch
$ORACLE_HOME/sqlpatch/sqlpatch.pl
use strict;
use Getopt::Long;
use sqlpatch.pm

Datapatch | Patch Apply Sequence
datapatch
1 Java patches
2 Bundle patches
3 One-off patches

Datapatch | Patch Rollback and Apply Queue
Binary Registry after opatch: datapatch queue
Patch 444 – Java Patch Rollback:
Patch 555 – Bundle Patch Apply:
Patch 666 – One-off Patch Rollback:
Patch 222 to 555 – Bundle

Cumulative: Patch
SQL Registry before datapatch:
Patch 111 – Java Patch Apply:
Patch 222 – Bundle Patch $ ./datapatch
Patch 333 – One-off Patch

Datapatch | Rollback Script
Apply/rollback scripts:
$ORACLE_HOME/sqlpatch/.../nnn_apply.sql
$ORACLE_HOME/sqlpatch/.../nnn_rollback.sql
Rollback scripts (zipped as BLOB):

SELECT PATCH_DIRECTORY
FROM REGISTRY$SQLPATCH

Multitenant
CDB
PDB1 PDB3 PDB5 PDB7

CDB$ROOT
PDB2 PDB4 PDB6 PDB8
• Datapatch patches CDB$ROOT and PDB$SEED automatically
• Datapatch only patches open PDBs
• Datapatch determines parallel degree based on CPU count

Significantly speed up patching
using AutoUpgrade
• Applies to multitenant databases on RAC only

Distributed Patching
NODE 1
PDB1 PDB3 PDB5 PDB7

CDB$ROOT
PDB2 PDB4 PDB6 PDB8
NODE 2

NODE 1
PDB1 PDB5
CDB$ROOT
PDB2 PDB6
NODE 2
PDB3 PDB7
PDB4 PDB8

To enable distributed patching
$ cat RACCDB.cfg
upg1.source_home=/u01/app/oracle/product/19/dbhome_19_18
upg1.target_home=/u01/app/oracle/product/19/dbhome_19_19
upg1.sid=RACCDB
upg1.tune_setting=proactive_fixups=true,distributed_upgrade=true
$ java -jar autoupgrade.jar -config RACCDB.cfg -mode deploy

41%
In benchmark, time saved
by using distributed PDB patching
• 2 node RAC database

• 4 CPUs each
• CDB with 8 PDBs

By default,
AutoUpgrade uses two nodes
Copyright © 2023, Oracle and/or its affiliates

173
Leverage more nodes
$ cat RACCDB.cfg
upg1.source_home=/u01/app/oracle/product/19/dbhome_19_18
upg1.target_home=/u01/app/oracle/product/19/dbhome_19_19
upg1.sid=RACCDB
upg1.tune_setting=proactive_fixups=true,distributed_upgrade=true,active_nodes_limit=n
$ java -jar autoupgrade.jar -config RACCDB.cfg -mode deploy

Less components,
Faster patching
Typical candidates:
• JAVAVM
• ORDIM
• SDO

175
Highest Impact
OWM CONTEXT OLS XOQ

Workspace Manager Oracle Text Label Security Oracle OLAP API
DV APS
Data Vault Analytical Workspace
XDK
Oracle XDK
JAVAVM ORDIM SDO

JServer JAVA VM Oracle Multimedia Spatial Data Option

176

Patching Basics
Patching Methods
Quarterly Patching
Patching Insights
Summary

Upgrade to Always patch
Oracle Database 19c out-of-place Apply
Data Pump
Bundle Patch
Keep DB and GI Remove OJVM
patch level in sync if not in use
Use out-of-place Less components, Avoid downtime with

Apply faster patching RAC Rolling Patching
patching with
patches
a brand-new
regularly
Oracle Home
Apply
Always use
Release Updates
Significantly speed up the latest OPatch
Complete a rolling patch and MRPs
patching using
as soon as possible Distributed Patching
Use Fleet Patching & Provisioning
Use OPatch to
STARTUP UPGRADE
remove
not needed
inactive Comply with Recompile invalid objects
for patching
patches Maximum Availability Architecture before invoking datapatch

Photo by Kyle Johnson on Unsplash
Recorded Web Seminars
https://MikeDietrichDE.com/video
s
More than 30 hours of technical content,

on-demand, anytime, anywhere

YouTube | Oracle Database Upgrades and Migrations
• 300+ videos
• New videos every week
• No marketing
• No buzzword
• All tech
Link

THANK
YOU
Visit our blogs:
https://MikeDietrichDE.com
https://DOHdatabase.com
https://www.dbarj.com.br/en

2
THANK
YOU
Webinars:
https://MikeDietrichDE.com/videos
YouTube channel:
OracleDatabaseUpgradesandMigrations

3
THANK
YOU
From SR to Patch
Insights into the Oracle Database
Development Process
June 22, 2023 – 16:00 CEST

4
THANK
YOU

5

Oracle 23c 02

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Oracle 23c 02

Uploaded by

Copyright:

Available Formats

Oracle Database

Photo by JESHOOTS.COM on Unsplash

1 Copyright © 2023, Oracle and/or its affiliates

2 Copyright © 2023, Oracle and/or its affiliates

3 Copyright © 2023, Oracle and/or its affiliates

4 Copyright © 2023, Oracle and/or its affiliates

5 Copyright © 2023, Oracle and/or its affiliates

6 Copyright © 2023, Oracle and/or its affiliates

More than 30 hours of technical content,

7 Copyright © 2023, Oracle and/or its affiliates

Oracle Database Installation and Patches

Grid Infrastructure Patching

8 Copyright © 2023, Oracle and/or its affiliates

9 Copyright © 2023, Oracle and/or its affiliates

10 Copyright © 2023, Oracle and/or its affiliates

LONG TERM INNOVATION

Long Term Support Extended

Long Term Support release to the

12 Copyright © 2023, Oracle and/or its affiliates

• Oracle Database 19c

13 Copyright © 2023, Oracle and/or its affiliates

14 Copyright © 2023, Oracle and/or its affiliates

15 Copyright © 2023, Oracle and/or its affiliates

possible!! Oracle Database 18c

16 Copyright © 2023, Oracle and/or its affiliates

17 Copyright © 2023, Oracle and/or its affiliates

19 Copyright © 2023, Oracle and/or its affiliates

Oracle Database Installation and Patches

Grid Infrastructure Patching

20 Copyright © 2023, Oracle and/or its affiliates

21 Copyright © 2023, Oracle and/or its affiliates

22 Copyright © 2023, Oracle and/or its affiliates

• Use the Patch Download Assistant MOS Note: 2118136.2

23 Copyright © 2023, Oracle and/or its affiliates

24 Copyright © 2023, Oracle and/or its affiliates

a recent Release Update

25 Copyright © 2023, Oracle and/or its affiliates

• Always use Important Recommended One-Off Patches: MOS Note: 555.1

26 Copyright © 2023, Oracle and/or its affiliates

A collection of recommended one-off fixes

27 Copyright © 2023, Oracle and/or its affiliates

2021 2022 2023 2024

28 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4 MRP5 MRP6

19.18.0 19.18.0 MRP1 MRP2 MRP3 MRP4 MRP5 MRP6

19.19.0 19.19.0 MRP1 MRP2 MRP3 MRP4 MRP5 MRP6

19.20.0 19.20.0 MRP1 MRP2 MRP3 MRP4

19.21.0 19.21.0 MRP1

29 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1

30 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2

31 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2 MRP3

32 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4

19.18.0 19.18.0 MRP1

33 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4 MRP5

19.18.0 19.18.0 MRP1 MRP2

34 Copyright © 2023, Oracle and/or its affiliates

19.17.0 19.17.0 MRP1 MRP2 MRP3 MRP4 MRP5 MRP6

19.18.0 19.18.0 MRP1 MRP2 MRP3

35 Copyright © 2023, Oracle and/or its affiliates