This watermark does not appear in the registered version - http://www.clicktoconvert.

com

Aspiring to be a RAC DBA ? Part I

Ashok Singh Lead DBA , Fastenal ,Winona MN-55987

This watermark does not appear in the registered version - http://www.clicktoconvert.com

Objectives
§ § § § Importance of RAC Introduction to RAC Shared Storage Cluster Manager/Cluster Ready Services § Overview of Installation § Background Processes § Features of different releases

This watermark does not appear in the registered version - http://www.clicktoconvert.com

Most Appropriate

1 R RI 8 VHU V

This watermark does not appear in the registered version - http://www.clicktoconvert.com

Importance of RAC ?

§ § § § § §

Total Cost of ownership True Scalability Ease Of Administration High Availability Transparent to users Foundation for GRID computing

This watermark does not appear in the registered version .com Introduction to Real Application Cluster § § § § § § More than one instance per database. Instances are running on different nodes(hybrid).http://www.clicktoconvert. Nodes may have multiple CPU’s Instances write to the same physical Database Common data files and Control file on a shared storage Shared storage may be a Clustered file system or use raw devices or use Automatic Storage Management § Instances cummunicate over the Interconnect § Successor to Oracle Parallel Server (OPS) .

com Introduction to RAC § Every Instance will have its own redo log files and Undo segments § Every Instance has its own background process § All Cache (Data.This watermark does not appear in the registered version .clicktoconvert.http://www. . § Backup .Recovery and Instance Recovery are slightly different than single-instance § Users can connect to any active instance § Sessions failover using Transparent Application Failover (TAF) § Additional waits due to interconnect traffic .Library .Data Dictionary ) are synchronized by Cache Fusion .

com Introduction to RAC .http://www.This watermark does not appear in the registered version .clicktoconvert.

This watermark does not appear in the registered version .clicktoconvert.com Physical Layout of RAC .http://www.

com Maximum Availability Architechture .This watermark does not appear in the registered version .http://www.clicktoconvert.

Polyserve ….ASM is a good choice .clicktoconvert.This watermark does not appear in the registered version .com Shared Storage § § § § § § Simplified when compared to OPS Most companies have SAN in place Cluster Files System (CFS) / Raw devices CFS provides ease of administration CFS can be OCFS .http://www. Starting 10g .

2.This watermark does not appear in the registered version .clicktoconvert.1-4.http://www. archived log files -install ‘fileutils-4.i386.rpm’ for new cp and dd -even if one is using ASM can still use OCFS for trace files .com Oracle Cluster File System § OCFS -free for Linux and windows -has a GUI tool OCFSTOOL -Integrates well with RAC -data files. control files.

clicktoconvert. Be Careful while choosing disks for a Disk Group. . A Disk can be a partial.full or a LUN from the RG . A diskgroup is a combination of disks . ASM instance has no Data Dictionary .control and log files) Also useful for Non-RAC databases A new instance type -ASM is introduced in 10g. I/O is spread evenly across all disks of a diskgroup.This watermark does not appear in the registered version .http://www. ASM instance needs to be the first to start on any node.com Automatic Storage Management § § § § § § § § § Replacement for CFS (datafiles .

normal.create a dummy directory Use of ASM diskgroup is very simple › create tablespace asmtbl datafile '+ASMTEST' size 750m § .clicktoconvert. When combined with OMF increases managibility Remember –ASM cannot maintain empty directories › “delete Input” has issues .This watermark does not appear in the registered version .com Automatic Storage Management § § § § Disks can be dynamically added to any diskgroup. Additional redundancy level -external.high.http://www.

This watermark does not appear in the registered version . › › › › ASMCMD [+] > ls –l ASMCMD [+] > cd usr01 ASMCMD [+usr01] > cd TESTG2 ASMCMD [+usr01/TESTG2] > ls › › ASMCMD [+usr01/TESTG2] > lsct ASMCMD [+usr01/TESTG2] > lsdg .clicktoconvert.http://www.com ASM contd § Use srvctl to configure ASM as a required resource › › › › › crs_stat -p |grep -i REQUIRED srvctl modify instance -d PICKY -i PICKY1 -s +KSM1 srvctl config asm -n racnode1 srvctl status asm -n racnode1 crsctl check cssd § 10gR2 introduces asmcmd which makes administration very easy .

ora parameters § In short .clicktoconvert. • If using EMC powerpath ASMlib need not be installed • Introduces three additional background processes – › RBAL. ARBx and ASMB § ASM instance has it own set of v$views and init.This watermark does not appear in the registered version .http://www.only RMAN can be used with ASM.com ASM contd § Enterprise Manager can also be used for administering Disk Groups • At present .ASM is amazing ! .

.. Checking CRS health. CRS integrity check passed. . Checking existence of GSD node application (optional) Check passed. cluvfy stage -pre crsinst -n all cluvfy stage -pre dbinst -n all ……………………… ……………………… Checking daemon liveness. CRS health check passed.clicktoconvert.http://www.. Checking daemon liveness. Liveness check passed for "CSS daemon".com Cluster Verification § § § § § § Cluvfy is a command line tool for RAC users for verification Cluvfy will not adversely affect the system or cluster stack Cluvfy works as an oracle user Cluvfy can be installed from a zip file or used from the CD .. Checking existence of ONS node application (optional) Check passed..... Liveness check passed for "CRS daemon". Liveness check passed for "EVM daemon". Checking existence of VIP node application (required) Check passed.This watermark does not appear in the registered version .. Checking node application existence. Checking daemon liveness..

com Cluster Manager § § § § § § § § § Oracle Cluster Manager (9i) : Linux and Windows ORACM runs at the OS level on all the nodes Accepts registration of Oracle Instance Cluster Manager is installed on all the nodes Uses Hangcheck timer.This watermark does not appear in the registered version . Checks the health of the cluster at every Hangcheck tick Hangcheck timer replaces Watchdog daemon Implemented as a kernel module so it is much less affected by system load Uses quorum disk (voting ) to evict a node GSD –performs manageability tasks for the databases .clicktoconvert.http://www.

ocrdump.OCRSD and EVMD Uses OCR to store instance information Uses voting disk to maintain the cluster GSD still exists to store 9i instances Uses VIP for enhanced failover to increase HA Needs to installed in a separate home Procedure to stop /start CRS Use of ocrcheck .full stack clusterware Primarily responsible for HA architecture Uses OCSSD. ocrconfig – showbackup .http://www.com Cluster Ready Services (10g) § § § § § § § § § § Available for all OS.This watermark does not appear in the registered version .clicktoconvert.

http://www.clicktoconvert. § Use oifcfg to change and store in OCR the new IP .com Cluster Ready Services § Change mesg_logging_level = 3 in the $ORACLE_HOME/crs/srvm/admin/ocrlog. § CRS manages VIP used by clients to connect to RAC § CRS will relocate the VIP address of the failed node to a surviving node. § Clients may not notice this failover § To VIP back –the node will need a reboot or instance start or re-start of the node applications .This watermark does not appear in the registered version .ini for detailed logging regarding OCR .

71 182.dbf /dev/emcpowere6 -> Voting Disk /dev/raw/raw1 /dev/emcpowere5 /dev/raw/raw2 /dev/emcpowere6 › Starting 10g Release 2 the ocr and Voting disks can be mirrored .com racnode1 -> Public racnode2.com racnode1 racnode1pr -> Private racnode2.clicktoconvert.14.fastenal.168.fastenal.252.14.dbf /dev/emcpowere5 -> OCR #/dev/raw/voting.fastenal.fastenal.12 racnode1.fastenal.70 182.69 182.com racnode2 vracnode1.14.com Preparation for CRS § § § Three IP addresses are required per node of RAC Public .com racnode2 racnode2pr § Entries in the /etc/sysconfig/rawdevices #/dev/raw/ocr.168.68 182.11 182.Private and Virtual A typical host file will look like – 182.com racnode2v racnode1.http://www.com racnode1v -> VIP vracnode2.252.215.252.215.fastenal.This watermark does not appear in the registered version .252.14.

ora and listener.http://www.ora should use the VIP instead of the public IP address .com Virtual IP § Starting 10g CRS requires VIP for enhanced failover. § VIP is automatically configured by VIPCA (next slide) at the end of the CRS installation § Do not bring this interface UP before the installation . § VIP uses a secondary public IP address for HA § Without VIP client will wait for a TCP timeout ie a longer duration in the event of a failure § If VIP is configured right and a node fails then this VIP is automatically moved to another active node in the cluster § Tnsnames.clicktoconvert.This watermark does not appear in the registered version . .

252.8.168.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:16 eth0:2 Link encap:Ethernet HWaddr 00:11:25:8D:50:F8 inet addr:172.16.com VIP Failover eth0 Link encap:Ethernet HWaddr 00:11:25:8D:50:F8 inet addr:172.255 Mask:255.0 Mb) Interrupt:16 eth0:1 Link encap:Ethernet HWaddr 00:11:25:8D:50:F8 inet addr:172.16.This watermark does not appear in the registered version .8.clicktoconvert.255.43 Bcast:172.6 Mb) Interrupt:16 .0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:16 eth1 Link encap:Ethernet HWaddr 00:11:25:8D:50:F9 inet addr:192.16.255 Mask:255.16.42 Bcast:172.16.40 Bcast:172.255.215.11.8.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:92425253 errors:0 dropped:0 overruns:0 frame:0 TX packets:59137596 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:3479471028 (3318.252.7 Mb) TX bytes:742048244 (707.215.16.11.255.168.11.255 Mask:255.13 Bcast:192.http://www.255.255 Mask:255.255.252.2 Mb) TX bytes:191895149 (183.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2289758 errors:0 dropped:0 overruns:0 frame:0 TX packets:2228847 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:889997978 (848.

clicktoconvert.com Installing CRS .http://www.This watermark does not appear in the registered version .

com While Installing CRS-VIP .This watermark does not appear in the registered version .clicktoconvert.http://www.

This watermark does not appear in the registered version .clicktoconvert.http://www.com While installing CRS-VIP (contd) .

http://www.This watermark does not appear in the registered version .com Installing CRS .clicktoconvert.

This watermark does not appear in the registered version .http://www.clicktoconvert.com Installing CRS .

ASM2.racnode4.LISTENER_RACNODE4.racnode4.ons` on member `racnode4` succeeded.racnode4.racnode4. racnode4:oracle> crs_stop ora.racnode4.racnode4.racnode4.vip Attempting to stop `ora.racnode4.http://www.lsnr STATE=ONLINE on racnode4 NAME=ora.asm NAME=ora.lsnr` on member `racnode4` succeeded.LISTENER_RACNODE4.ons Attempting to stop `ora.This watermark does not appear in the registered version .LISTENER_RACNODE4.racnode4.ons STATE=ONLINE on racnode4 NAME=ora.ons` on member `racnode4` Stop of `ora.racnode4.racnode4.lsnr` on member `racnode4` Stop of `ora.gsd STATE=ONLINE on racnode4 NAME=ora.lsnr Attempting to stop `ora.racnode4.racnode4.clicktoconvert.gsd Attempting to stop `ora.vip` on member `racnode4` succeeded.com Adminstering CRS § Stopping CRS: racnode4:oracle> crs_stat |grep racnode4 STATE=ONLINE on racnode4 NAME=ora. racnode4:oracle> crs_stop ora.racnode4.gsd` on member `racnode4` succeeded. racnode4:oracle> crs_stop ora.racnode4.vip STATE=ONLINE on racnode4 racnode4:oracle> crs_stop ora.LISTENER_RACNODE4.gsd` on member `racnode4` Stop of `ora.racnode4.vip` on member `racnode4` Stop of `ora. .racnode4.

This watermark does not appear in the registered version .week.…. % ocrconfig -showbackup › › Backup is done only on the node which node master Backup is kept in the $ORA_CRS_HOME/cdata/<cluster name> § Day.clicktoconvert. .Configuration tool for Oracle Cluster Registry.http://www.com Administering CRS § OCRCHECK : Status of Oracle Cluster Registry is as follows : Version : 2 Total space (kbytes) : 409496 Used space (kbytes) : 4592 Available space (kbytes) : 404904 ID : 1833543341 Device/File Name : /dev/raw/raw1 Device/File integrity check succeeded Device/File Name : /dev/raw/raw2 Device/File integrity check succeeded Cluster registry integrity check succeeded § § § % ocrdump ${HOST}_OCRDUMP ocrconfig .

Install cluster Manager (9i) first .com/technology/pub/notes/ technote_singh_crs.http://www.Install 10g CRS + Oracle Binaries Use srvctl from 9i to manage 9i DB and from 10g to manage 10g DB § Very helpful for test & dev environments § For details refer to my technical note on OTN http://www.oracle.This watermark does not appear in the registered version .clicktoconvert.html . Stop GSD but keep ORACM still executing Now .com Installing 9i and 10g together ! § § § § § With a little extra effort it can be done .

TCP/IP…protocol .clicktoconvert.low CPU consumption and high transfer rate(millions/second) § It can be either Gigabit ethernet.com Interconnects § Critical component of RAC § Uses the private network for communication with other nodes § Do not use the private IP address for any other purpose. VIA … and use UDP. § Connected via a switch to other nodes § Can use a cross-over cable for two nodes in a test environment . § This enhancement in technology has helped Cache Fusion .http://www. § Ideally –low latency .This watermark does not appear in the registered version .

2 from 9i .215.0 oradebug setmypid oradebug ipc exit § sqlplus> connect / as sysdba The trace file which will tell about the Interconnect used .168.This watermark does not appear in the registered version .1.11 § From Alert Log : › › cluster interconnect IPC version:Oracle UDP/IP IPC Vendor 1 proto 2 Version 1.com Interconnects § Alert file contains more info about Interconnect § New wait events due to traffic over interconnect § Interconnect may use the public network after you perform the upgrade to 10. › cluster_interconnects 192.0.clicktoconvert.http://www.

§ Use DBCA or create manually § Create database at least once manually –to understand things better § Configure listeners .enterprise manager .clicktoconvert.com Oracle Database § Installation of Oracle Binaries is easy .log and v$resource_limit for more learning § Explore $ORACLE_HOME/srvm/admin for some scripts § Upgrading from 9i to 10g › CRS has to be upgraded first. § Use srvctl to register the database in the OCR § Check alert.http://www.This watermark does not appear in the registered version .. .

http://www.com Things are going fine ! .This watermark does not appear in the registered version .clicktoconvert.

http://www.clicktoconvert.com Some new Parameters § § § § § § § § § Instance_number Thread Cluster_database Cluster_database_instance Cluster_interconnects Remote_listener Local_listener Parallel_instance Instance_group NB :Some parameters need to have identical values on all Nodes To change a value :”alter system set open_cursors=1000 sid=‘*’ scope=both “ .This watermark does not appear in the registered version .

clicktoconvert. . .global enqueues calculated at startup § LMD :Manages enqueues and updates GRD.This watermark does not appear in the registered version .Co-ordinates with remote LMD’s § DIAG :Captures diagnostic data regarding process failures.http://www.monitors LMD . .performs recovery in case of failure .com Background Processes § LMON: GES monitors global enqueues and resources between nodes.

clicktoconvert. builds a read-consistent image if required satisfies requests from remote LMS locates. .20(10g) depends on the messaging (9i cpu_count/4 10g gcs_server_processes) Co-ordinates activity on data blocks ensures that updates are performed on the latest block.http://www.com Background Processes § ü LMSn: (GCS) Up to 10(9i).This watermark does not appear in the registered version .prepares and transmits a data block ü ü ü ü ü § LCK: handles resource requests not related to nonCache Fusion operations such as instance requests and cross-instance call operations.

.com SRVCTL § SRVCTL – Is used by many tools eg EM Easy to use to perform configuration on the cluster Read/writes information from the OCR Calls sqlplus on each node when required Extensive help srvctl status service -h Usage: srvctl status service -d <name> [-s "<service_name_list>"] [-f] [-v] [-S <level>] -d <name> Unique name for the database -s "<serv.http://www.clicktoconvert..>" Comma separated service names -f Include disabled applications -v Verbose output -S <level> Additional information for EM Console -h Print usage .This watermark does not appear in the registered version ..

http://www....com Some Examples of SRVCTL § § § § § § § § § § § § Srvctl –v --Should be executed from the correct ORACLE_HOME Do not cancel srvctl as it may corrupt the configuration data srvctl stop instance -d CORPREPO -i "CORPREPO1.CORPREPO2" -o immediate srvctl start database –d CORPREPO –o mount srvctl status database –d CORPREPO srvctl config Srvctl stop listerner Srvctl relocate –services from one node to another Srvctl modify database –d WMST –r physical_standby Srvctl status nodeapps –n racnode1 Srvctl enable/disable database/instance/service .This watermark does not appear in the registered version .clicktoconvert.-i ”wmst1.wmst2” Srvctl remove service/instance/database ..

none(default) -method –basic or preconnect -retries and delay § Load_balance :YES.http://www.failed_over § New Connections depends upon Node load.This watermark does not appear in the registered version .failover_method.Instance load and dispatchers (if MTS) .OFF.com Transparent Application Failover § § § § § Transparent to users.clicktoconvert.defined in tnsnames Service name is used instead of SID Service will have many listeners and instances Supports load-balancing and failover Failover modes (defined in connect_data) -type ---session.select.NO.TRUE § GV$SESSION –failover_type.

application workloads can be defined as services.dbms_services or and Enterprise Manager § Can always query gv$active_services view to explore more about services (max 64 services) › Two internal services –sys$background and sys$users § Creation of service will add an entry in the TNS.This watermark does not appear in the registered version .http://www. § Services can be managed individually § Instances are divided into PREFERRED and AVAILABLE § Entire database load can be broken into services § Reduces the need to manage users and sessions § Services have more significance in RAC environment § Services can be created using DBCA .clicktoconvert. .com Services in Oracle 10g § Starting Oracle 10g ..

clicktoconvert.http://www.com Configuring Services .This watermark does not appear in the registered version .

This watermark does not appear in the registered version .clicktoconvert.http://www.com Configuring Services .

com Services racnode2:oracle> srvctl start service -d WEBTST -s webtest racnode2:oracle> srvctl status service -d WEBTST -s webtest -f -v Service webtest is running on instance(s) WEBTST1 racnode2:oracle> srvctl stop instance -d WEBTST -i webtst1 racnode2:oracle> srvctl status service -d WEBTST -s webtest -f -v Service webtest is running on instance(s) WEBTST2 racnode2:oracle> srvctl config service -d WEBTST webtest PREF: WEBTST1 AVAIL: WEBTST2 .clicktoconvert.http://www.This watermark does not appear in the registered version .

http://www. § TAF and FCF should not be used together .clicktoconvert.balancing across Listeners Server Side – Listener directs to the best instance Load Balancing Advisory :Directs the new connections LBA is used by listener.This watermark does not appear in the registered version .JDBC implicit Connection Cache Feedback is inbuilt by using THROUGHPUT or SERVICE_TIME For enhanced HA RAC uses Fast Application Notification Easy Integration of Fast Connection Failover for HA § Driver Independence § Integration with implicit Connection Cache § Enable FCF and forget it .com Load Balancing § § § § § § § Client Side -.

› › › If service on PREF instance fails it moves to AVAIL instance.This watermark does not appear in the registered version .2 and aq_ha_notifications must be set to TRUE .com Connection Load Balancing § CLB_GOAL is set using EM or dbms_service for any service § Long– Long-lived connections(default) § Short –Short-lived connections § 10gR2 introduces Server side TAF while using Services. Use dbms_service.clicktoconvert. Client and Server must be 10.http://www.modify_service to change parameters .

com Different Releases § Oracle 9i –RAC introduced § OCFS for linux and windows users § Oracle 9.watchdog timer replaced by hangcheck timer .clicktoconvert.0.NET § Oracle 10gR2 – › › › › .4 Very stable release .2. § Oracle 10gR1— › › › Cluster Manager replaced by CRS Concept of Services expanded ASM introduced ASM improved CLUVY OCR and Voting disks can be mirrored Can use FAN/FCF with TAF for OCI and ODP.http://www.This watermark does not appear in the registered version .

com Configuring Services .clicktoconvert.http://www.This watermark does not appear in the registered version .

This watermark does not appear in the registered version .http://www.clicktoconvert.com Question and Answers ? ? ? Thank You ! .

Sign up to vote on this title
UsefulNot useful