You are on page 1of 49

Oracle 12c

Flex ASM & Flex Cluster


PRESENTER
Yenugula Venkata Ravikumar
OCM, Head-Database Practice, Patterns

8th NOVEMBER 2014


BENGALURU, INDIA

Patterns 1
ABOUT ME
Patterns 2
AGENDA

The History of ASM


Benefits Of ASM
Automatic Storage Management (ASM) Overview
Flex ASM
Flex Cluster
Oracle 10g R2 Functionality
Oracle 11g R2 Functionality
Oracle 12c R1 Functionality
ASM Cluster
ASM Network
ASM Remote Access
Deploying the Flex ASM

Patterns 3
HISTORY OF ASM
Before Oracle 10g, Storage Management was provided by 3rd parties

CUSTOMER CHALLENGES
Large databases required considerable storage management
THE SIMPLE
BestIDEA OF ASM required raw storage
performance
NFS solutions,
Integrated while simple,
volume manager didsystem
and file not perform as well as raw
For
Stripe RAC,
and cluster
mirror filesfile systems
across diskswere not available
in a ASM Disk Group
Automatic Rebalance
With on
Built Oracle
the 10g, ASM
Oracle provided
instance architecture
Simplicity of management,
Manage storage as a global cluster even betterDisk
Shared thanGroups
NFS solutions
Performance equal to raw disks
Support for RAC clusters

Patterns 4
BENEFITS OF ASM (1 of 2)

I/O is spread evenly across all available disk drives to prevent hot spots and
maximize performance
Inherent large file supports
Performs automatic online redistribution after the
incremental addition or removal of storage capacity
Maintains redundant copies of data to provide
high availability or leverage 3rd party RAID functionality
Supports Oracle Database as well as Oracle Real Application Clusters (RAC)

Patterns 5
BENEFITS OF ASM (2 of 2)

For simplicity and easier migration to ASM, an Oracle Database can contain
ASM and non-ASM files. Any new files can be created as ASM files whilst
existing files can also be migrated to ASM
RMAN commands enable non-ASM managed files to be relocated to an ASM
disk group
Oracle Enterprise Manager can be used to manage ASM disk and file
management activities
ASM reduces Oracle Database cost and
complexity without compromising
performance or availability

Patterns 6
ARCHITECTURE OF ASM

ORACLE DATABASE Oracle Database


ASM
File System
File System & Volume Management
Logical Volume Manager
OPERATING SYSTEM OPERATING SYSTEM

HARDWARE HARDWARE

Patterns 7
Oracle RAC 10gR2 CONFIGURATION

Patterns 8
Oracle RAC 11gR2 CONFIGURATION

Patterns 9
CAPABILITIES OF ASM

1-1 ASM to SERVER

SHARED DISK GROUPS

WIDE FILE STRIPING

Patterns 10
AUTOMATIC STORAGE MANAGEMENT (ASM) OVERVIEW
Current State

RAC CLUSTER
Database Instance

ONE TO ONE DBA


ASM Instance
DBA DBB DBB DBB DBC
MAPPING OF ASM ASM Instance

INSTANCES TO
SERVERS Node1 ASM Node2 ASM Node3 ASM Node4 ASM Node5 ASM

ASM Cluster Pool of Storage


SHARED DISK GROUPS Disk Group A Disk Group B ASM Disk

WIDE FILE STRIPING

Patterns 11
Oracle RAC 11gR2 CONFIGURATION
Current Challenge Oracle RAC 11gR2 CLIENTS

Loss of Service
If ASM on any node fails
Database service is down WAN TRAFFIC

+ASM1 +ASM2 +ASM3 +ASM4

4 Node
Primary Site
Patterns 12
Oracle Database 11gR2 with ASM

LOG INTO RAC DATABASE INSTANCE


login as: oracle (RAC1)
oracle@192.168.xx.xxs password: ********
Last login: Fri Sep 27 06:05:44 2013
CHECK STATUS OF ASM & RAC DATABASE
[oracle@rac1 ~]$ ps -ef | grep pmon INSTANCES
oracle 3053 1 0 05:56 ? 00:00:00 asm_pmon_+ASM1
oracle 3849 1 0 05:57 ? 00:00:00 ora_pmon_flavia1
CHECK STATUS OF ASM INSTANCE IN RAC
[oracle@rac1 ~]$ srvctl status asm DATABASE INSTANCES1 (RAC1)
ASM is running on rac2,rac1
CHECK STATUS OF CLUSTER IN RAC
[oracle@rac1 ~]$ crsctl check cluster
DATABASE INSTANCES1 (RAC1)
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online STOP THE ASM INSTANCE IN
RAC DATABASE INSTANCE1
[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f (RAC1)

Patterns 13
Oracle Database 11gR2 with ASM

CHECK THE STATUS OF ASM INSTANCE IN


RAC DATABASE INSTANCE1 (RAC1)

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f


[oracle@rac1 ~]$ srvctl status asm

Patterns 14
Oracle Database 11gR2 with ASM

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f


[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2

Patterns 15
Oracle Database 11gR2 with ASM

CHECK
[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f THE STATUS OF ASM & RAC
[oracle@rac1 ~]$ srvctl status asm DATABASE INSTANCE (RAC1)
ASM is running on rac2
[oracle@rac1 ~]$ ps -ef | grep pmon

Patterns 16
Oracle Database 11gR2 with ASM

SUMMARY
The database instance is strongly linked to the ASM instance
If an ASM instance fails so will the database instance on the
same node

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f


[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2
[oracle@rac1 ~]$ ps -ef | grep pmon
oracle 7885 5795 0 06:20 pts/0 00:00:00 grep pmon

Patterns 17
FLEX ASM: ELIMINATE 1:1 SERVER MAPPING

New: ASM Storage Consolidation in Oracle Database 12c

RAC Cluster
Database Instance

DATABASES SHARE DBA


ASM Instance
DBA DBB DBB DBB DBC
ASM INSTANCES ASM Instance

Node1 Node2 ASM Node3 ASM Node4 ASM Node5

Node1
Node1 Node2 Node5
runs
runs as
as runs as runs as
ASM
ASM Client
Client ASM Client ASM Client
to
to Node4
Node2 to Node3 to Node4
ASM Cluster Pool of Storage
SHARED DISK Disk Group A Disk Group B ASM Disk

GROUPS

WIDE FILE STRIPING

Patterns 18
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ hostname LOG INTO RAC DATABASE


oel6-112-rac1.localdomain
[oracle@oel6-112-rac1 Desktop]$ ps -ef | grep pmon
INSTANCE 1 (RAC1)
oracle 3325 1 0 17:39 ? 00:00:00 asm_pmon_+ASM1
oracle 3813 1 0 17:40 ? 00:00:00 mdb_pmon_-MGMTDB CHECK THE STATUS OF ASM & RAC
oracle 5806 1 0 17:42 ? 00:00:00 ora_pmon_orcl1 DATABASE INSTANCES
oracle 6193 1 0 17:42 ? 00:00:00 apx_pmon_+APX1
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm CHECK THE STATUS OF ASM INSTANCE IN
ASM is running on oel6-112-rac2,oel6-112-rac1 RAC DATABASE INSTANCE FROM
INSTANCE 1 (RAC 1)
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online CHECK THE STATUS OF CLUSTER IN
CRS-4533: Event Manager is online INSTANCE 1 (RAC 1)
[oracle@oel6-112-rac1 Desktop]$ asmcmd
ASMCMD> showclustermode CHECK IF ORACLE FLEX ASM IS ENABLED
OR NOT (RAC1)

Patterns 19
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ asmcmd


ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate

Patterns 20
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ asmcmd


ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate
Normal
ASMCMD> exit

Patterns 21
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ asmcmd


ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate
Normal
ASMCMD> exit

[oracle@oel6-112-rac1 Desktop]$ srvctl config asm -detail

Patterns 22
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ asmcmd


ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate
Normal
ASMCMD> exit
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm -detail
ASM is running on oel6-112-rac2,oel6-112-rac1 ASM is enabled.
[oracle@oel6-112-rac1 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
CHECK IF ORACLE FLEX ASM IS ENABLED
ASM listener: LISTENER ASM is enabled.
OR NOT (RAC2)
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac2 Desktop]$ asmcmd

Patterns 23
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ asmcmd


ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate
Normal
ASMCMD> exit
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm -detail
ASM is running on oel6-112-rac2,oel6-112-rac1 ASM is enabled.
[oracle@oel6-112-rac1 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac2 Desktop]$ asmcmd
ASMCMD> showclustermode
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate

Patterns 24
Oracle Database 12cR1 with FLEX ASM

Normal
ASMCMD> exit

Patterns 25
Oracle Database 12cR1 with FLEX ASM

CHANGE CARDINALITY OF THE ASM


(RAC 2)

Normal
ASMCMD> exit
[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail

Patterns 26
Oracle Database 12cR1 with FLEX ASM

Normal
ASMCMD> exit
[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac2 Desktop]$

Patterns 27
Oracle Database 12cR1 with FLEX ASM

Normal
ASMCMD> exit
[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid BRING DOWN THE ASM INSTANCE IN
Password file: +DATA/orapwASM RAC DATABASE INSTANCE 1 (RAC 1)
CHECK STATUS OF ASM INSTANCE IN RAC
ASM listener: LISTENER ASM is enabled. DATABASE INSTANCE1 (RAC1)
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM CHECK STATUS OF CLUSTER SERVICES IN
RAC DATABASE
[oracle@oel6-112-rac1 Desktop]$ srvctl stop asm -node oel6-112-rac1 INSTANCE1
-stopoption abort (RAC1)
-force
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm
PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate with crsd
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster

Patterns 28
Oracle Database 12cR1 with FLEX ASM

Normal
ASMCMD> exit
[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac1 Desktop]$ srvctl stop asm -node oel6-112-rac1 -stopoption abort -force
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm
PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate with crsd
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

Patterns 29
Oracle Database 12cR1 with FLEX ASM

Normal
ASMCMD> exit
[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail
ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac1 Desktop]$ srvctl stop asm -node oel6-112-rac1 -stopoption abort -force
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm
PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate with crsd
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster
CHECK STATUS OF ASM & RAC DATABASE
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online IN INSTANCE1 (RAC 1)
CRS-4533: Event Manager is online
[oracle@oel6-112-rac1 Desktop]$ ps -ef | grep pmon

Patterns 30
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac2 Desktop]$ srvctl config asm -detail


ASM home: /u01/app/12.1.0/grid
Password file: +DATA/orapwASM
ASM listener: LISTENER ASM is enabled.
ASM instance count: 3
Cluster ASM listener: ASMNET1LSNR_ASM
[oracle@oel6-112-rac1 Desktop]$ srvctl stop asm -node oel6-112-rac1 -stopoption abort -force
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm
NOTE
PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate with crsd
Herecommunicate
a database instance is associated
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster
CRS-4535: Cannot with Cluster with the specific ASM instance running
Ready Services
in theSynchronization
CRS-4529: Cluster specific node. IfServices
in case due to some reason if the ASM instance was
is online
unable
CRS-4533: Event to beisbrought
Manager onlineup/services goes down, still the database instance can be
[oracle@oel6-112-rac1 Desktop]$ ps -ef | grep pmon
oracle 3813brought1 up
0 as the database
17:40 ? instance
00:00:00will look for ASM instance running in the
mdb_pmon_-MGMTDB
oracle 5806same cluster.
1 0 17:42 ? 00:00:00 ora_pmon_orcl1
oracle 6193 1 0 17:42 ? 00:00:00 apx_pmon_+APX1
[oracle@oel6-112-rac1 Desktop]$

Patterns 31
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac2 Desktop]$ . oraenv


ORACLE_SID = [orcl1] ? orcl1
ORACLE_HOME = [/home/oracle] ? /u01/app/oracle/product/12.1.0/db_1
The Oracle base remains unchanged with value /u01/app/oracle

CHECK STATUS OF RAC DATABASE


INSTANCE RUNNING WITHOUT ASM
INTANCE IN RAC DATABASE INSTANCE1
(RAC1)

Patterns 32
Oracle Database 12cR1 with FLEX ASM

LOG INTO DATABASE INSTANCE FROM


[oracle@oel6-112-rac1 Desktop]$ sqlplus /nolog
SQL*Plus: Release 12.1.0.1.0 Production on Wed Sep 25 18:24:36 2013 Copyright RAC 2013,
(c) 1982, DATABASE INSTANCE
Oracle. (RAC1)
All rights reserved.

SQL> connect sys/oracle@orcl as sysdba


Connected.
SQL> select instance_name, instance_number from gv$instance;
INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl2 2
orcl1 1
SQL> select instance_name, instance_number from v$instance;
INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl2 2
SQL> connect sys/oracle@orcl as sysdba
Connected.
SQL>

Patterns 33
Oracle Database 12cR1 with FLEX ASM

SQL> select instance_name, instance_number from gv$instance;


INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl1 1
CONNECTING TO ASM INSTANCE OF RAC
DATABASE INSTANCE2 (RAC2) FROM RAC
[oracle@oel6-112-rac1 Desktop]$. oraenv DATABASE INSTANCE1 (RAC1)
ORACLE_SID = [orcl1] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/12.1.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@oel6-112-rac1 Desktop]$ asmcmd --privilege sysasm --inst +ASM2

Patterns 34
Oracle Database 12cR1 with FLEX ASM

ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB
Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 15342 4782
0 4782 0 Y DATA/
ASMCMD>

SUMMARY
The database instance was using a dedicated ASM instance and that
ASM instance was forced to stop working simulating a failure, so the
database instance reconnected to an existent ASM instance on another
node, for this example node 2 (rac2).

Patterns 35
FLEX ASM: SUPPORTING ORACLE DATABASE 11g

Previous Database versions will host Local ASM Instance

RAC CLUSTER
Database Instance

11.2 11.2
DATABASES SHARE DBA
ASM Instance
DB DBA DBB DBB DBB DB DBC
ASM INSTANCES

Node1 ASM Node2 ASM Node3 ASM Node4 ASM Node5 ASM

ASM Cluster Pool of Storage


SHARED DISK GROUPS Disk Group A Disk Group B ASM Disk

WIDE FILE STRIPING

Patterns 36
Oracle FLEX CLUSTER
Comprises of a Hub and Leaf architecture Depicts a typical Oracle flex cluster with four Leaf nodes and two Hub
nodes. In a nutshell Oracle Flex Cluster requires Oracle Flex ASM.
where in only the Hub nodes will only have
direct access to Oracle Cluster Registry
(OCR) and Voting Disk (VD)
However application can access the
database via Leaf nodes without ASM
instance NOT running on Leaf nodes
The connection to the database is through
Hub making it transparent for the
application
Patterns 37
HUB NODES & LEAF NODES
Hub Nodes
They are connected among them via private network and have direct access to
the shared storage just like previous versions. These nodes are the ones that
access the Oracle Cluster Registry (OCR) and Voting Disk (VD) directly.

Leaf Nodes
These nodes are lighter and are not connected among them, neither access the
shared storage like the Hub Nodes. Each Leaf Node communicates with the Hub
Node that is attached to, and its connected to the cluster via the Hub Node that
is linked to.
Patterns 38
FLEX ASM & FLEX CLUSTER
Flex ASM Flex CLUSTER
Higher Availability (DB Less complex configuration and management for
instances can use remote ASM non-database nodes
Less complex configuration for Non-Database
instances)
nodes, such as Processing Nodes, Application /
Planned or Unplanned
Web Server Nodes etc.
downtime
Reduces per-node ASM
instance overhead
ASM only needs to run on 3
nodes in the cluster
Facilitates DB, Cluster & server
consolidation
Multiple DBs & applications in
a cluster
Patterns 39
PREVIOUS ASM CLUSTER

ASM architecture utilized an ASM


instance on every server
Database instances dependent on
node-specific ASM instance
ASM overhead scaled with size of
cluster

Patterns 40
NEW ASM CLUSTER
Eliminates requirement for an ASM instance on every cluster
server
Database instances connects to any ASM
instance in the cluster
Database instances can failover to a
secondary ASM instance
Administrators specify the cardinality of ASM
instances (default is 3)
Clusterware ensures ASM cardinality is maintained
Patterns 41
FLEX ASM NETWORK
Flex ASM adds the ASM network, which is used for
communication between ASM and its clients
Oracle 12c, you can now create a separate network
dedicated for ASM communication.
Need to mention the network address during the
installation
This dedicated ASM network is completely optional.
The default is the pre-12c behavior of using the
Clusterware interconnect for ASM communication
Patterns 42
DEPLOYING FLEX ASM
Managing Flex ASM
Flex ASM requires minimal monitoring and ongoing management
Primary objective is that instances are up and running
SRVCTL Command for:
Checking ASM instance status
Setting cardinality
Starting Stopping - and - Relocating ASM instances
No Flex ASM-specific instance parameters are required
Default settings will effectively support most situations
ASM server instances use automatic memory management
Flex ASM is managed by ASMCA, CRSCTL, SQL*Plus and SRVCTL
Patterns 43
NEW FLEX ASM

Flex Features
Increase maximum number of Disk Groups to 511 (Previous
limit was 63)
Command for renaming ASM Disk
ASM Relocate Command
Clients are automatically relocated to another instance if
an Oracle ASM instance fails
When necessary, Clients can be manually relocated
using the ALTER SYSTEM RELOCATE CLIENT command

Patterns 44
New Option in Grid Infrastructure

Oracle ASM Deployment Model can be


chosen during the installation of
Oracle Grid Infrastructure
Oracle ASM Configuration Assistant
(ASMCA) can be used to enable Oracle
Flex ASM after the installation /
upgrade is performed
The functionality is available only in an
Oracle Grid Infrastructure
Configuration

Patterns 45
Conversion of Standard Cluster to
Oracle Flex ASM Cluster

Patterns 46
Patterns 47
CHECK CLUSTER MODE AND CLUSTER STATE AFTER
LOGGING INTO ASMCMD PROMPT
[oracle@racnroll1 ~]$ asmcmd
ASMCMD> showversion
ASM version : 12.1.0.1.0
ASMCMD>showclusterstate
Normal
ASMCMD>showclustermode
ASM cluster : Flex mode enabled

Patterns 48
Thanks for your time!
Q&A Session
Refer for additional information
yenugulavenkata.ravikumar
http://docs.oracle.com/
http://www.oracle.com/technetwork/es/articles/index.html
ravikumar.yv@patterns-sdi.com

@yvrk1973

http://yvrk1973.blogspot.in

http://in.linkedin.com/pub/yv-ravikumar-oracle-certified-master-ocm/14/13/a50
Patterns 49

You might also like