Dbvisit v11 Oracle

Technical training

2022
Agenda

1) Dbvisit V11 Architecture and Concept

2) Prerequisites
3) Installation & Configuration
4) Create Standby database (CSD) pre-checks and process
5) Synchronization
6) Archive log Management
7) Notifications
8) Switchover
9) Failover
10) DR Testing
11) Upgrade
12) User Roles
Jan Klinke
Presales Consultant and Product
Specialist @ Dbvisit
jan.klinke@dbvisit.com
Vijay Sivaprakasam
Head of Customer Services
vijay.sivaprakasam@dbvisit.com
#1 Architecture
Basic Concept

● All Dbvisit processes run in stacks (as opposed to v10 and before: dbvnet,
dbvagent, dbvctl, observer …)

● Only two types of stack: Dbvagentmanager and Controlcenter

● Dbvagentmanager is installed on every host, Controlcenter is single entity

● Dbvagentmanagers communicate with controlcenter

● If Controlcenter is down the basic tasks like synchronization still continue

ARCHITECTURE OVERVIEW
How Standby works
with Oracle Database

Archived logs are transferred via Standby

product to standby server and there
applied to standby database

Transfer is encrypted, can work with

lower bandwidth and higher response
time networks

In case of network interruption archive

logs transfers simply queue up and are
sent one the standby site is available Automated Archivelog Standby in
again operation Apply Delay MOUNT

Archivelog shipping and It is possible to specify Standby database is at all

apply are managed by archive log apply delay times in MOUNT mode and
daemon processes (transfer still happens can’t be accessed for
including forced logswitch immediately) reading
on primary
 Considerations

● It’s required to install ControlCenter

● As of now Running with command line is entirely possible except create DDC and
setting automated standby update

● In the future we plan to support command line only approach

#2 Prerequisites
 Supported OS & DBs

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3238264943/System+Requirements

v11 Supported on:

● Linux 6-8 (RHEL, SLES, OEL), please don’t use non-certified systems like CentOS
●
●
Windows 2008-2022
Standby Version 10.1
Refer to MOS for certified OS

v11 Supporting:

●
Demo
Oracle 10g - 21c (Latest Patchset recommended) EE+SE2
● 19.17 !!! - requires 11.2.1 Hotfix

Primary & Standby Equivalency

There can be differences in OS versions. Database

versions must be identical. No cross-platform!

©2021
© 2021Dbvisit
DbvisitSoftware
Software ||dbvisit.com
dbvisit.com
 Dbvisit works and is supported with major
Dbvisit and
cloud providers
Cloud
• Operating system access needed
• Cloud only and also hybrid deployments
NETWORK OVERVIEW https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3238264943/System+Requirements

DbvagentManager Ports

Listens on:
7890 TCP File Transfer

ControlCenter Ports

Listens on:
4433 TCP Webserver
5533 TCP Agent Incoming
Communication
 Installation Requirements

Storage Space

● Reserve at least 15GB for v11 Installation Base

● Mainly for traces, switchover
● Installed Oracle Home on standby server identical to primary
● Shared volume & dedicated VIP for SEHA and RAC installations

Network

● FW Rules as per diagram

● Hostname Resolution Equivalency, Date & Time synchronized

Users

● DbvagentManager should be same as who owns Database Software

● No special requirement Control Center user
#3 Installation &
Configuration
 Standby Installation

● By default listen on all IPs

● Remember and use the password for Dbvagentmanager installations
● Default Options:

Install Directory: /usr/dbvisit

Control Center Web Server Port: 4433
Agent Communications Port: 5533
Agent File Transfer Port: 7890
● Systemctl service can be created after installation:

sudo /usr/dbvisit/standbymp/bin/dbvcontrol service install --user oracle

sudo /usr/dbvisit/standbymp/bin/dbvagentmanager service install --user oracle

● Possible to also run in background:

nohup /usr/dbvisit/standbymp/bin/dbvagentmanager service run &

nohup /usr/dbvisit/standbymp/bin/dbvcontrol service run &
 Considerations

● Dbvagentmanager will be visible in CC under the hostname you enter during

installation (be consistent with FQDNs)

● If you run with systemctl, consider turning off selinux and be aware of setting limits, as
it will be in fact dbvagentmanager process starting the database (all limits are
inherited) for example:

...
[Service]
LimitNOFILE=65536
LimitNPROC=16384
LimitMEMLOCK=infinity
...

● Batch install possible:

./install-agent-v11.2.0 -batch
 Creating Oracle Configuration

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3258253344/Create+Standby+Multipl
atform+Configuration

Archive Log Files Location on

Source Host

Temporary staging area for

archivelog shipping for Dbvisit
NOT db_file_recovery_dest nor
log_archive_dest

Archive Log Files Location on

Standby Host

Destination directory for archivelogs

transferred from primary server
 Oracle Configuration File

● Complete reference at:

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3287482402/DDC+File+Refere
nce

● Text file dbv_<NAME>.env located in /usr/dbvisit/standbymp/oracle/conf

● Can be modified by direct editing by any text editor or using ControlCenter

● Some settings are stored locally, some in ControlCenter

● Located on primary and standby server. On Primary is “master” file, on standby it’s
only a copy

● Not all variables are listed although all of them are used
#4 CSD Pre-checks and
process
 Creating Oracle Standby Database - Prechecks

● Primary Database in FORCE LOGGING mode

SQL> select force_logging from v$database;

● Primary database in archivelog mode

SQL> archive log list

● All Datafiles must be ONLINE / SYSTEM

● Disable any backup jobs (deleting of archivelogs during create process isn’t good)

● Disk Space (default compressed backup ~ 20-30% of the total database size)

● Redo log size and switch frequency (primary database should be switching logfile
about 5-6 times per hour)

● Redo log size doesn’t matter, archivelog doesn’t contain empty space
 SQL for log switches

select to_char(trunc(l.first_time,'hh24'),'yyyy-mm-dd hh24:mi') date_time

, count(l.recid) archivelog_count
, round(sum(l.blocks*l.block_size)/1024 /1024) MB
from v$archived_log l
where l.first_time > sysdate-14
group by to_char(trunc(l.first_time,'hh24'),'yyyy-mm-dd hh24:mi')
order by date_time desc;
 Creating Oracle Standby Database - File Names Prechecks

● Check names of all database files:

SQL> select name from v$datafile;

SQL> select member from v$logfile;
SQL> select name from v$tempfile;

● Database file names should not be a mix of non-OMF and OMF naming (OMF=Oracle
Managed Files)
OMF= /u02/app/oracle/oradata/MEGA/datafile/o1_mf_cwc_kcy4tyky_.dbf
non-OMF= /u02/app/oracle/oradata/MEGA/datafile/users01.dbf

● When using ASM, file names should be exclusively OMF

● When using filesystem, we can use OMF convention or (exclusive or) non-OMF
convention (never mix)

● When using ASM and OMF datafiles on primary need to be located on single
diskgroup. It’s not possible to create standby DB with datafiles in +DATA1, DATA2 etc …
 Creating Oracle Standby Database

● Creates RMAN backup of primary database, transfers it to standby server and

restores there (all in parallel)

● By Default RMAN backup is compressed (optimized for WAN networks)

● No Impact on primary (except increased I/O)

● Don’t forget to modify standby Oracle Instance parameters

● v11 Supports new features:

- Use Shared Storage between primary and standby (/backup/db)

- archivelogs shipping is started as soon as the create standby database process starts

● By default synchronization is enabled after process is finished making whole process

as simple as “start before going home from the office, check next day”
#5 Synchronization
Automatic Synchronization of Standby Database

● Is asynchronous and done via applying primary archivelogs to the standby database

● On primary the archivelog is copied directly from FRA or log_archive_dest via network
to the ARCHDEST directory on the standby server

● Dbvisit processes check each 5 seconds on primary and standby for new archivelog

● In case FRA or log_archive_dest is located on ASM, Dbvisit will first copy the
archivelog to the local filesystem directory ARCHSOURCE (DDC parameter)

● Dbvisit Synchronizes all operations including addition of datafiles and even whole PDB

● In case of archivelog loss and unresolvable gap it’s possible to synchronize standby
database with the RMAN incremental backup
 Enabling Automatic Synchronization

● Is enabled by default and synchronization starts the moment standby database

creation is started

● Default synchronization interval is 10 minutes or as soon as the primary database

creates new archivelog on its own

● Automatic synchronization can be only controlled from GUI:

$ ./dbvctl -d MEGA -D start

Dbvisit Daemon is now controlled automatically through the Control Center.

# Manual Synchronization on primary & on standby works

./dbvctl -d MEGA

● It is possible to schedule manual synchronization in crontab / Windows Scheduler for

pseudo - automatic synchronization
 Archivelog apply Mode

● sqlplus - default, very fast for applying individual archivelogs. Very slow for applying
large amount of archivelogs

● RMAN - can be set by specifying DDC parameter APPLY_ARCHIVE_RMAN = Y. Very

slow for applying individual archivelogs, but much faster than sqlplus when applying
large amounts of archivelogs at once
 RMAN Incremental backup sync

● Can be used anytime

● Will resolve issue with nologging transactions and also resolve issue when archivelogs
needed to recover standby database are for any reason deleted from the primary or
when there are too many archivelogs to apply

● It is needed to provide temporary space for the backup on primary and standby server

● Backup is only delta between standby datafiles and primary datafiles

● In some cases when standby database is lagging far behind Incremental backup
doesn’t make much sense and it’s better to recreate standby DB from scratch
RMAN Incremental backup sync - example
#6 AMM
 Automatic Archive Log Management (AMM)

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3349217327/Archivelog+Management+Module

● Using Dbvisit AMM for archivelog housekeeping is optional on both sides (it’s disabled by
default.

● Can be enabled separately for primary and standby server

● Dbvisit will never delete an archive which was not yet transferred or applied

● AMM is triggered by the process which transfers / applies archivelogs - the transfer / apply
must be successful, otherwise AMM is not triggered

● Only two criteria for deletion: based on time and based on total count of archivelogs

● On Primary: AMM deletes archivelogs from FRA (db_recovery_file_dest) or log_archive_dest

● On Standby: AMM deletes archivelogs from Dbvisit directory ARCHDEST
 AMM on Primary

● AMM can be set using GUI or Edit DDC file directly

● AMM on primary example settings in DDC file:

ARCHSOURCE_MANAGEMENT = Y <<enable AMM on primary>>

DAYS_TO_KEEP_ARCHSOURCE = 0.5 <<age of archivelogs in days that are retained in FFRA/log_archive_dest>>
NUM_ARCHSOURCE_TO_KEEP = 0 <<number of archivelogs in days that are retained in FRA/log_archive_dest>>
ARCHSOURCE_BACKUP_COUNT = 1 <<number of compulsory RMAN backups for archivelogs>>
THRESHOLD_ARCHSOURCE = 100 <<percentage full threshold FRA/log_archive_dest disk % for emergency delete>>
FRA_THRESHOLD_ARCHSOURCE = 100 <<%full threshold of FRA for emergency delete (db_recovery_file_dest_size)>>
DELETE_ARCHSOURCE_THRESHOLD = N <<enable emergency delete (N = alert only)>>

● There is logical AND between criteria

● Dbvisit NEVER deletes archivelog which was not yet transferred to standby
 AMM on Standby

● AMM on standby example settings in DDC file:

ARCHDEST_MANAGEMENT = Y <<enable AMM on standby>>

DAYS_TO_KEEP_ARCHDEST = 2 <<age of archivelogs in days that are retained in ARCHDEST>>
NUM_ARCHDEST_TO_KEEP = 0 <<number of archivelogs in days that are retained in ARCHDEST>>
THRESHOLD_ARCHDEST = 100 <<percentage full threshold of ARCHEST for emergency delete>>
DELETE_ARCHDEST_THRESHOLD = N <<enable emergency delete (N = alert only)>>

● There is logical AND between criteria

● Dbvisit NEVER deletes archivelog which was not yet applied to standby
#7 Notifications
Notifications Global Settings Setup

● Global Email Notifications settings include general information about SMTP server valid for
ALL configurations and ALL databases

● Possible to use TLS as well as unencrypted SMTP communication

Notifications Settings for Individual DDCs

● For receiving notifications we need to change the individual DDC settings:

Specify email addresses of recipients:

Enable Observer to send mails:

 API

● There’s API available for easy integration with existing monitoring systems

● API is not suitable for other actions such as switchover, create standby database etc …

# on webserver host:

curl -k -u admin:admin -d '{}' https://localhost:4433/api/dashboard | jq

#8 Switchover
 Graceful Switchover

● Controlled and reversible process for switching roles of Primary and Standby DB with Zero data loss

● Can be done only when in full and complete control over the whole environment

● Downtime required for both primary and standby

● Takes about 10-20 minutes to complete (Database size is not a factor)

● Can be triggered via GUI or CLI:

./dbvctl -d <DDC> -o switchover

● With Dbvisit process to switchover and getting back is still the same process, all variables are updated
automatically
 Graceful Switchover - Preparation

● Stop synchronization in Control Center

● Stop Application and Listener

● Restart primary database (this ensures no sessions will remain in database and prevent shutdown)

● Make sure no logswitches will be done on primary (for example scheduled backups)

● Perform one manual archivelog send & apply

● Perform switchover
#9 Failover
 Failover

● Activation of standby database (opening read write) with no interaction with Primary whatsoever

● Non-reversible action

● Can be triggered via GUI or CLI:

./dbvctl -d <DDC> -o activate

● Intelligent activation (when delay is set) only in GUI

● Possibility to check with read only test before activation

Failover & Recovery from Failover
#10 DR Testing
 DR Testing

● Take a backup of the standby database (Image copy or backupset)

● Activate the standby database (opens the database in read-write resetlogs)

● Connect to your application and check application integrity

● Reinstate standby database from the backup taken previously

■ Drops the activated standby database

■ Restores the standby from the backup

■ Opens in mount mode

● Resume the automated standby database to sync the standby database with primary.

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3347677186/Oracle+Activation+and+DR+Testing#2.-Test-Standby-Activation
#11 Upgrade
 Upgrade from v10/v9

1) Stop your Dbvisit Schedules and processes

2) Backup the Dbvisit Base location where your software is installed.

3) Download the latest version from www.dbvisit.com

4) Extract the install files into a temporary folder, example /home/oracle/MP

5) Start the Installer for required component

6) Import DDC into ControlCenter

7) Apply the new version 11 license key.

8) Run a manual send/apply of logs once

9) Enable the automatic standby update and cleanup

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3258220610/Upgrading+from+v9+or+v10
#12 User Roles
 User Roles in Control Center

● There are three main roles in control center

○ Admin

○ Operator

○ Monitor

https://dbvisit.atlassian.net/wiki/spaces/DSMP/pages/3391324188/User+Management
Thank
You
dbvisit.com/getstandby

jan.klinke@dbvisit.com
info@dbvisit.com