Professional Documents
Culture Documents
SFRAC:
• Supports a maximum of 16 nodes
• Monitors an Oracle instance running on
each system simultaneously
• Manages concurrent access to shared
storage containing database files
• Maintains cluster membership and
provides membership to Oracle
clusterware
Clients
Cluster
interconnect Fibre SAN
Switches
Storage
Clients
Custom
VCS application clusters
BEA
Replication
SF RAC
SFRAC provides the infrastructure required
Clusterware and RAC Extensions
by Oracle RAC:
• Cluster-enabled versions of the industry standard VCS
Veritas storage management platform
ODM Library
• Optimizations for Oracle Grid or Clusterware and
Oracle RAC Cluster File System
• Certification by Oracle
Cluster Volume Manager
Server
VCS core
G L L G
ODM A L Data file management L A ODM
B T T B
CFS File system metadata CFS
vxconfigd
GAB vxfen HAD
kmsg
GLM GMS
VCSMM
LMX
UDP
a b h v w u y f d o
GAB
LLT
Ethernet
Other instances/cluster members
14 InfoScale Enterprise VSE+ training
I/O fencing
• Fencing:
̶ Allows access to storage by cluster vxfen GAB vxfen
members only
̶ Denies write access to non-members
• The fencing driver, vxfen: SAN
vxfen
CVM
CVM
• Port v: Kernel messaging
kmsg vxconfigd – Node membership information
– Master node selection/re-selection
GAB v w u y • Port w: vxconfigd configuration daemon messaging
LLT
VxVM shared disk group configuration changes propagate
LLT across the cluster:
GAB v w u y – vxassist
kmsg vxconfigd – vxdg
• Port u: vxconfigd commands shipped from slave to master node
• Port y: Shipping I/O when node loses access to shared storage
CVM
Port f: GLM
CFS
• Primarily for cache coherency
GLM • GLM mastered per file system
VCSMM
GAB f • Locking modes:
LLT ̶ Shared
̶ Update
LLT
GAB f ̶ Exclusive
• Cluster-wide cache coherency
GLM
• Bypassed by RAC ODM in SFRAC
CFS
Agents
Agents
VCS core
25 InfoScale Enterprise VSE+ training
Thank you!
Copyright © 2019 Veritas Technologies, LLC. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies or its affiliates
in the U.S. and other countries. Other names may be trademarks of their respective owners.
This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or
implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.
InfoScale Enterprise
VSE+ Level Training
Database instance 1
• Eliminates double buffering
System Global Area (SGA) • Avoids file system single write locks
DB buffer cache • Supports kernel asynchronous I/O
Processes
Buffer cache
ODM
libodm
Port d ODM:
GMS
• Oracle Managed Files (OMF) communication
GAB d
̶ Identifies Oracle data files
LLT
̶ Is used when creating and deleting files
LLT
GAB d • Group Message Service (GMS) adds cluster capability to
the ODM library
GMS
libodm
ODMODM
VCSMM
• Port o: VCSMM
GAB o
LLT • skgxn
LLT
– Makes ioctl calls to VCSMM
GAB o – Is provided as a linked library with SFRAC
VCSMM
libskgxn
Oracle
libskgxp libskgxp
10g 11g
Oracle
35 InfoScale Enterprise VSE+ training
Oracle Database concepts
Data Files
Redo Files
Control Files
Oracle Database
Server
Client 1 Listener 2 4 Database
process
Extents
VxVM/VxFS
db_block_size blocks
A checkpoint operation causes all modified blocks of data in the buffer cache to be
written to disk.
Data:
– Is written to buffers for improved performance.
– Is not guaranteed to be in the data file until a checkpoint occurs.
Checkpoint:
– Is written to the redo log buffer and flushed to disk when completed.
– Indicates the point from which Oracle recovers if the instance fails.
OCI client
Oracle Net Services
Listener
Data Files
Oracle
Redo Files
ODM
Control Files
VxFS /u02
Archive Files
VxVM usr2dg
Oracle Database
OCI client
Oracle Net Services
Listener
Data Files
Oracle
Redo Files
ODM ODM
Control Files
VxFS /u02 VxFS
Archive Files
VxVM usr2dg VxVM
Oracle Database
OCI client
Oracle Net Services
Listener
Data Files
Oracle
Redo Files
ODM ODM
Control Files
VxFS /u02 VxFS
Archive Files
VxVM usr2dg VxVM
Oracle Database
OCI client
Oracle Net Services
Listener
Data Files
Oracle
Redo Files
ODM ODM
Control Files
VxFS /u02 /u02 VxFS
Archive Files
VxVM usr2dg usr2dg VxVM
Oracle Database
OCI client
Oracle Net Services
Listener
Data Files
Oracle
Redo Files
ODM ODM
Control Files
VxFS /u02 VxFS
Archive Files
VxVM usr2dg VxVM
Oracle Database
Copyright © 2019 Veritas Technologies, LLC. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies or its affiliates
in the U.S. and other countries. Other names may be trademarks of their respective owners.
This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or
implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.
InfoScale Enterprise
VSE+ Level Training
Data Files
Control Files
Clients
Oracle instance 2
Archive Files
Oracle Database
ONS ONS
Server Server
Oracle instance 1 Oracle instance 2
System Global Area System Global Area
Database Shared Redo log Database Shared Redo log
buffer pool buffer buffer pool buffer
cache cache
Processes Processes
Processes Processes
Processes Processes
Database
A
A→A’
Example:
If the DBA performs change such as modifying structure of a table or index on one
node, the other RAC instances must be notified and prevented from making
concurrent changes to those structures.
Checkpoint:
• Records are written to the redo log buffer of each instance when certain events occur.
• Records of instance 1 are written to Thread A of the redo log on disk.
• Records of instance 2 are written to Thread B of the redo log on disk.
• Are written at different points in time for each instance.
Threads:
• Are implemented as separate files.
• With the last checkpoint record, indicates the point from which Oracle recovers if that instance
fails.
Note: At checkpoint time, any modified blocks in the buffer caches are written to the data files on disk.
Note:
• Redo logs must be located on either CFS or shared raw devices.
• Archive logs must be on CFS file systems.
NIC
CRS Arb.
• VCS starts instance
LISTNR • CRS starts listener
INST1 INTERCON Cluster control
MNT • Start and stop of CRS and RAC instances
DISK • Controls RAC cluster membership
• Disk infrastructure monitoring and remediation
• Monitors database instance and listener health
DB Instance
DB Instance
ARCn ARCn
CKPT Database CKPT
DBWn DBWn
Data files
Redo logs
ODM ODM
Archive logs
CFS Disk I/O CFS
Control files Disk I/O
CVM CVM
Shared concurrent access storage
GCS GCS
RAC Instance
RAC Instance
LMD LMD
LCK Cache fusion LCK
Lock management
LMS LMS
IPC IPC
70 InfoScale Enterprise VSE+ training
Veritas Storage Foundation for Oracle RAC
OCI client
Oracle Net Services
Listener Listener
and VIPs and VIPs
CRS/Grid CRS/Grid
Cluster interconnect
Instance Instance
ODM ODM
Data files
Redo logs
Archive logs
Control files
• Automatic or manual management of database instances can be performed outside VCS control, if
desired.
• Private VIPs for clusterware heartbeats and Cache Fusion:
– Are managed by VCS for 10g
– Are managed by Oracle clusterware for 11g
CVMVxconfigdArgs syslog
Sample
Resource definition value
• Starts, stops, and monitors CFS:
Service group cvm
– vxfsckd daemon
– GAB port f Resource name vxfsckd
Resource type CFSfsckd
• Is managed by the CFSfsckd agent Key attributes
None
cssd_resource
Resources are added to cvm to manage
Oracle clusterware:
ora_priv
• Oracle cssd daemon
ocrvote_cfsmount
cvm_vxconfigd
redo_cfsmnt
• Contains CVMVolDg and CFSMount resources
for managing Oracle data
orasg
• Requires setting database to manual startup
oradata_voldg
• Failure behavior
– cvm faults, orasg is taken offline
– orasg faults, cvm remains online
cvm
IP
NIC
Mount CFSMount
Mount CFSMount
Mount CFSMount
DiskGroup
Oracle RAC SG
Copyright © 2019 Veritas Technologies, LLC. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies or its affiliates
in the U.S. and other countries. Other names may be trademarks of their respective owners.
This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or
implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.