11gR2 Oracle Real Application Clusters / Grid Infrastructure N.F.

René Kundersma (rene.kundersma@oracle.com)

Database Technical Architect – Oracle Technology Services
 The following is intended to outline our general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any
features or functionality described for Oracle’s
products remains at the sole discretion of Oracle.

About me…
René Kundersma, Software Ingenieur

• 1998: Sun Microsystems

• 2001: Qualogy Consultancy
• 2006: Oracle Technology Services (Oracle Consulting), The
Netherlands

Oracle Certified Master 9i/10g, Certified Expert RAC/Linux

Currently working on first Dutch Database Machine
implementation (v1 & v2)

Working with 11gR2 RAC since September 2007

New Features / Agenda
Answer most common questions about
Oracle Clusterware and Oracle RAC

• Upgrading to Oracle RAC 11g Release 2 <Insert Picture Here>

• New De-Install Utility

• Managing Your Oracle RAC 11g Release 2
database
• SCAN
• Multiple Public Networks
• Managing OCR and Voting Disks with ASM
(ASMCF)
• Grid Plug and Play
• RAC One Node

<Insert Picture Here>

Upgrading to Oracle RAC

11g Release 2
Upgrading to Oracle Database 11g Release 2

• Oracle Clusterware and Oracle Automatic Storage

Management are a single binary distribution called
“grid infrastructure”
– Out of Place Upgrade (home is owned by ROOT)
– Recommended to be on local file system

• Upgrade Oracle Clusterware and Automatic Storage

Management to 11.2.0.1 at the same time
– ASM 11.1 only is rolling upgradeable

Introducing Grid Infrastructure

• Software provided by Oracle that is outside the
database
– I.E. infrastructure software: volume management, file system
management, cluster software
• Software generally managed by the System
Administrator (even if it comes from Oracle!)
• Only one copy runs on a server
– May have extra copy during out-of-place upgrade
• Grid Infrastructure for a Cluster
• Includes Oracle Clusterware, ASM
• Grid Infrastructure for a Standalone Server
• Includes Oracle Restart, ASM
• Need SCAN for upgrade
Mixed Database Versions in an 11.2 Cluster

• Oracle Database 10g and Oracle Database 11g are

supported with Oracle Clusterware 11g Release 2
• You must upgrade Oracle Clusterware and Automatic
Storage Management to 11.2.0.1 at the same time
• Apply DBCA patch Bug 8288940
• “Pin” the nodes
crsctl pin css -n nodename

Where did OPROCD go?

• OPROCD, OCLSOMON, OCLSVMON no longer exist

with Oracle Clusterware 11g Release 2
• Hangcheck timer is not required on Linux
• Code has been rewritten and functionality is handled
by Oracle Clusterware Agents.
 Oracle Clusterware Startup Sequence

<Insert Picture Here>

Managing Your Oracle RAC

11g Release 2 Database
 Two Management Styles
for Oracle RAC

• Administrator Managed
– Specifically define where the database should run with a list
of servers
– Define where services should run within the database

• Policy Managed
– Define resource requirements of workload
– Enough instances are started to support workload
requirements
– Goal: To remove hard coding of a service to a specific
instance or node

Server Pool

• Logical division of the cluster into pools of servers

• Managed by crsctl (applications), srvctl (Oracle)
• Defined by 3 attributes (min, max, importance) or a
defined list of nodes
– Min- minimum number of servers (default 0)
– Max – maximum number of servers (default 0 or -1)
– Importance – 0 (least important) to 1000

srvctl modify serverpool –g <name> –u <max>

 Server Pool Example – Instance View

public network
VIP1 VIP2 VIP3 VIP4
Scan_LSNR Scan_LSNR GNS Scan_LSNR
Listener Listener Listener Listener

instance 1 instance 2 instance 1 instance 2

ASM cluster ASM ASM ASM
interconnect
Oracle Clusterware Oracle Clusterware Oracle Clusterware Oracle Clusterware

Operating System Operating System Operating System

Operating System

backoffice shared storage frontoffice

Min 1 Max 2Imp 3 Min 1 Max 2 Imp 4

Redo / Archive logs all instances

Database / Control files
Managed by ASM
OCR and Voting Disks

Cluster Managed Services with Server Pools

• One to one mapping

– IE a service can only run in one server pool
• Services are uniform (run on all instances in the
pool) or singleton (runs on only one instance in the
pool)
Server Assignment for policy based management

• Servers are assigned in the following order:

1. Generic server pool
2. User assigned server pool
3. Free
• Oracle Clusterware uses importance of server pool
to determine order
1. Fill all server pools in order of importance until they meet
their minimum
2. Fill all server pools in order of importance until they meet
their maximum
3. By Default any left over go into FREE

Server Assignment
Cluster Management via Server Pools

Back Office Front Office LOB Free

Min 1 Max 3 Imp 3 Min 2 Max 3 Imp 4 Min 0 Max 2 Imp 2
Server Assignment
Cluster Management via Server Pools

Back Office Front Office LOB Free

Min 1 Max 3 Imp 3 Min 2 Max 3 Imp 4 Min 0 Max 2 Imp 2

Server Assignment
Cluster Management via Server Pools

Back Office Front Office LOB Free

Min 1 Max 3 Imp 3 Min 2 Max 3 Imp 4 Min 0 Max 2 Imp 2
 Cluster Reconfiguration

• If a server leaves the cluster Oracle Clusterware

may move servers from one server pool to another
only if a server pool falls below its min.
• It chooses the server to move from
– A server pool that is less important
– A server from a server pool of the same importance which
has more servers than its min
• Oracle Clusterware will only move servers if you
have non-default values for min, importance

GENERIC Server Pool

• Used to model databases that are “Administrator

Managed”. i.e. “The OLD WAY!”
– Parent of all server pools for “Administrator Managed”
databases
• Always exists but may be of size 0
• Used for upgrade from 10g or 11g Release 1
• Managed by srvctl managing the database
• Servers in ‘Generic’ are named (hosting member in
cluster resource)
Administrator Managed and Policy Managed
Cluster Management via Server Pools

GENERIC
Back Office Front Office LOB
11g R1 10g R2 11g R2
Min 0 Max 2 Imp 2

Notes for Policy-Managed Databases

• SIDs are DYNAMIC

– DBA scripts may have to be adjusted
– Environment variables settings in profiles will have to check
the current sid
– Directories for LOG files will change over restarts
• “Pin” Nodes
– Forces Oracle Clusterware to maintain the same Node
number when restarted (which maintains SID)
– Automatically done on upgrades
– Required when running pre-11g Release 2 versions in the
cluster
– On upgrade, the nodes will be “pinned”
crsctl pin css -n nodename
 <Insert Picture Here>

SCAN

Single Client Access Name (SCAN)

SCAN

Back Office Front Office LOB Free

• Used by clients to connect to any database in the cluster

• Removes the requirement to change the client connection if
cluster changes
• Load balances across the instances providing a service
• Provides failover between “moved instances”

© 2009 Oracle Corporation – Proprietary and Confidential

Network Requirements for SCAN

• Two options:
1. Define the SCAN in your corporate DNS (Domain Name
Service)
sales1-scan.example.com IN A 133.22.67.194
IN A 133.22.67.193
IN A 133.22.67.192

2. Use the Grid Naming Service (GNS) and the SCAN will be
created during cluster configuration

Single Client Access Name (SCAN)

• Allows clients to use EZConnect or simple JDBC

connections
• Each cluster will have 3 scan listeners, each having a
scanvip defined as cluster resources on network 1
• A SCAN VIP/LISTENER will failover to another node in
cluster
• Instance registers with local listener on its node
• Database “remote_listener” registers instances with all
SCAN listeners
sqlplus system/manager@sales1-scan:1521/oltp
jdbc:oracle:thin:@sales1-scan:1521/oltp
 Oracle Clusterware SCAN Resources

• Created by root script during Grid Infrastructure Installation

$srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1521
SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1521

$srvctl config scan

SCAN name: sales1-scan, Network: 1/192.87.58.0/255.255.255.0/
SCAN VIP name: scan1, IP: /sales1-scan.mycompany.com/192.87.58.143
SCAN VIP name: scan2, IP: /sales1-scan.mycompany.com/192.87.58.99
SCAN VIP name: scan3, IP: /sales1-scan.mycompany.com/192.87.58.100

SCAN Overview

Application Server

Oracle RAC
Database

SCAN
Listeners
Listeners

Clients
 SQL*NET and SCAN
• SQL*NET will retrieve the IP addresses from DNS or
GNS, it will then load balance and failover across the
IP addresses.
• For MAA implementations, if client uses both primary
and standby in address list, SQL*NET will retrieve all
6 IPs, it will then load balance and failover across the
6 IP addresses
sales.mycompany.com =(DESCRIPTION=
(CONNECT_TIMEOUT=10)(RETRY_COUNT=3)
(ADDRESS_LIST= (LOAD_BALANCE=on)(FAILOVER=ON)
(ADDRESS=(PROTOCOL=tcp)(HOST=scan1)(PORT=1521))
(ADDRESS=(PROTOCOL=tcp)(HOST=scan2)(PORT=1521)))
(CONNECT_DATA=
(SERVICE_NAME= sales.mycompany.com)))

<Insert Picture Here>

Grid Plug and Play (GPnP)

Grid Plug and Play

• Makes it easy to add, replace, or remove nodes in a

cluster
• Allow the cluster to manage it’s own virtual ip
addresses
– No need to go back to the network administrator

New Cluster Components for GPnP

• GPnP Profile for Global Configuration

– Provides cluster configuration information to allow a node to
join the cluster
• IP Multicast Service Discovery/mDNS
– Resolves names to addresses without /etc/hosts or DNS
configuration
• Grid Naming Service
– Lets the cluster manage it’s own network
– Support DHCP for IPs and VIPs
– No need to go back to the Network Admin
 Network Administrator Requirements GNS Zone
Delegation

• Create entries in the corporate DNS servers that will

delegate the authority for a portion of the network to
the cluster
– IE Let the cluster manage its own network requirements

# Delegate to gns on mycluster

mycluster.example.com. NS gns-vip.mycluster.example.com
#Let the world know how to go to the GNS vip
gns-vip.mycluster.example.com. 10.9.8.7

GPnP Architecture – Client Connections

4
1 client
DNS
Corporate
2 domain 5

Cluster Sub- GNS 6 Local

SCAN
domain listener listener
Back Office Front Office LOB Free

ps
c vi
a mi
dyn

DHCP DATA RECOV

server
Next Steps

Oracle RAC : http://otn.oracle.com/rac

Oracle XPS-ON-HA Blog: http://blogs.oracle.com/xpsonha/

Oracle RAC SIG: http://www.oracleracsig.org

Oracle Consulting Technology Services:

http://www.oracle.com/global/uk/expert_services/index.html

Rene.Kundersma@oracle.com

Q& QUESTIONS

A ANSWERS

http://otn.oracle.com/rac
For More Information

search.oracle.com
Real Application Clusters

or
otn.oracle.com/rac