Commvault Engineer

Course Guide

Module 3 – Virtualization

Commvault Platform Release 2023E

Notices and Disclaimers
Commvault, Commvault and logo, the "C hexagon” logo, Commvault Systems, Metallic, Metallic and logo, the “Wave”
logo, Commvault HyperScale X, HyperScale X, Recovery Reserve, and ThreatWise are trademarks or registered
trademarks of Commvault Systems, Inc. (“Commvault). The unauthorized use of any Commvault trademark is strictly
prohibited.
Other company and product names mentioned herein may be trademarks of their respective owners. References to
any third-party products, services, or websites should not be considered an endorsement by Commvault. Some
examples are for illustration only and are fictitious.
All right, title, and interest, including all intellectual property rights in and to this document and to any related subject
matter (collectively “Ownership Rights”) are owned and expressly reserved by Commvault. No Ownership Rights are
granted to you.
This document is intended for distribution to and personal reference use solely by Commvault customers. All use of
Commvault Solutions, including this document, is governed by Commvault’s Master Terms & Conditions (currently
available at [Link] which are incorporated herein in their
entirety.
This document is provided “as is.” Information in this document, including any specifications, URLs, or other
references, is subject to change without notice. See [Link]/IP for more information about our
trademarks, patents, and other IP rights.
Confidentiality
This document contains information that is confidential and proprietary to Commvault. Without limiting rights under
copyright or otherwise, this information is provided with the express understanding that it will be held in strict
confidence and that no part of this document will be disclosed, used, reproduced, stored, or transmitted, in whole or in
part, for any purpose other than as expressly approved or provided by Commvault in writing.

©1999-2023 Commvault

© 2023 Commvault.
Table of Contents
Virtualization .................................................................................................................................................................. 4
Transport Modes ........................................................................................................................................................... 6
Virtual Server Agent Backup Process ......................................................................................................................... 11
VSA Advanced Restore Options ................................................................................................................................. 19

© 2023 Commvault.
Virtualization

Virtualization is standard for data center consolidation, whether on-premises or in the cloud. A
comprehensive protection strategy is required to ensure proper protection. Commvault provides
several protection methods for virtual environments on premises and in the cloud. These methods
provide a comprehensive enterprise hybrid protection strategy. There are four primary methods
Commvault can use to protect virtual environments.

Virtual Server Agent (VSA)

The Commvault Virtual Server Agent (VSA) interacts with the hosting hypervisor to provide protection
at the virtual machine level and is often the preferred method. Agents do not need to be installed
directly on the virtual machines, although installing restore-only agents provides a simplified method
for restoring data back to the VM.
Depending on the hypervisor application being used and the virtual machine's operating system,
different features and capabilities are available. The VSA interfaces with the hypervisor's APIs and
provides capabilities inherent to the application. As hypervisor capabilities improve, the Commvault
VSA agent is enhanced to take advantage of new capabilities.

Agent-Based Protection
Agent-based protection uses Commvault agents installed directly in the virtual machine. When an
agent is installed in the VM, it appears in the CommCell console just like a regular client and the
functionality is the same as an agent installed on a physical host.
The main advantage with this configuration is that all the features available with Commvault agents
are used to protect data on the VM. For applications, using a Commvault agent provides complete
application awareness of all data protection operations including streaming log backups, granular
item-based protection, archiving and content indexing.

© 2023 Commvault.
VSA Application Aware Protection
VSA application aware backups insert a plugin into the VM during a VSA backup. When a VM backup
runs, the plugin quiesces the application using a VSS snapshot. The VSA coordinator then
communicates with the hypervisor to conduct a VM snapshot. This protection method provides a
hybrid approach using the VSA to conduct data protection jobs, and agent-based functionality for
recovery, similar to installing an agent directly in the VM.

IntelliSnap for VSA

The Commvault IntelliSnap feature provides integration with supported hardware vendors to conduct,
manage, and create backup copies of snapshots. This technology is used to snap VMs at the
datastore level and back them up to protected storage.
The process for protecting virtual machines is similar to performing snapshots with the VSA agent
directly interfacing with the hosting hypervisor application. The VSA first quiesces the virtual machine
and then the IntelliSnap feature uses vendor APIs to perform a hardware snapshot of the datastore.
The datastore is then mounted on a hypervisor proxy and all VMs are registered. Finally, the VMs are
backed up and indexes are generated for granular level recovery. The snapshots can also be
maintained for live browse and recovery. The backup copies are used for longer term retention and
granular browse and recovery.

© 2023 Commvault.
Transport Modes
The VMware VADP framework provides several transport modes to protect virtual machines.
Variables such as physical architecture, source data location, ESX resources, network resources and
VSA proximity to MediaAgents and storage influence the mode you choose. It is also recommended
to consult with Commvault for design guidance when deploying Commvault in a VMware
environment.

© 2023 Commvault.
SAN Transport Mode

SAN Transport Mode is used on a VSA proxy with direct Fibre Channel or iSCSI access to snapshot
VMs in the source storage location. This mode provides the advantage of avoiding network movement
of VM data and eliminates load on production ESX servers.
Virtual machines are backed up through the VSA and to the MediaAgent. If the VSA is installed on a
proxy server configured as a MediaAgent with direct access to storage, LAN-Free backups can be
performed. For best performance, Commvault recommends that the VSA have a dedicated HBA to
access the VMDK files. If an iSCSI SAN is used, we recommend a dedicated Network Interface Card
on the VSA for access to the SAN.

© 2023 Commvault.
HotAdd Mode

HotAdd mode uses a virtual VSA in the VMware environment. This requires all data to be processed
and moved through the VSA proxy on the ESX server. HotAdd mode has the advantage of not
requiring a physical VSA proxy and does not require direct SAN access to storage. It works by 'hot
adding' virtual disks to the VSA proxy and backing up the disks and configuration files to protected
storage.
A common method of using HotAdd mode is to use Commvault deduplication with client-side
deduplication, DASH Full and incremental forever protection strategy. Using Change Block Tracking
(CBT), only changed blocks within the virtual disk have signatures generated and only unique block
data are protected.
This mode is also useful when there is no physical connectivity between the physical VSA proxy and
the datastore storage preventing the use of SAN transport mode. Some examples of such scenarios
are when using NFS Datastores or using ESX hosts local disk storage to host datastores.

© 2023 Commvault.
NBD Mode

NBD mode uses a VSA proxy installed on a physical host. The VSA connects to VMware and
snapshots will be moved from the ESX server over the network to the VSA proxy. This method
requires adequate network resources. NBD mode is the simplest method to protect virtual machines.

© 2023 Commvault.
Hyper-V Transport Mode

Commvault uses VSA proxies to facilitate the movement of virtual machine data during Hyper-V
backup operations. The VSA proxies are identified in the instance properties. For Microsoft Hyper-V,
the VSA is installed on each hypervisor host. VMs can be protected from each host or a VSA proxy
can be designated to protect VMs. The proxy must have access to all clustered shared volumes
where VMs reside.

10

© 2023 Commvault.
Virtual Server Agent Backup Process

The VSA works by communicating with the hosting hypervisor to initiate software snapshots of virtual
machines. Once the VMs are snapped, the VSA backs them up to protected storage. The following
steps illustrate the process of backing up VMware virtual machines.
• Virtual Server Agent communicates with the hypervisor instance to locate virtual machines
defined in the subclient that requires protection.
• Once a virtual machine is located, the hypervisor prepares the virtual machine for the
snapshot process.
• The virtual machine is placed in a quiescent state. For Windows VMs, VSS is engaged to
quiesce disks.
• The hypervisor then conducts a software snapshot of the virtual machine.
• The virtual machine metadata is extracted.
• The backup process then backs up all virtual disk files and VM configuration files.
• Once the disks are backed up, indexes can optionally be generated for granular recovery.
• The hypervisor deletes the snapshots.

11

© 2023 Commvault.
Virtual Server Agent Proxy Roles

Virtual Server Agent (VSA) proxies are defined at the instance level of the VSA pseudo client. The top
listed VSA proxy is designated as the coordinator and all other proxies are designated as data
movers. Note that the coordinator proxy also acts as a data mover. The coordinator is responsible for
communicating with the hypervisor to get information about VMs and distribute VM backups to data
mover proxies. Data mover proxies communicate with the coordinator proxy and provide information
on available resources and job status. If the coordinator proxy is unavailable, the next proxy in the list
assumes the role of coordinator. If a data mover proxy becomes unavailable, the coordinator proxy
assigns jobs to other available proxies.

Virtual Machine Distribution Process

When a VSA subclient backup starts, the coordinator receives a list of all virtual machines listed in the
subclient. Based on a defined set of rules, the coordinator creates a dynamic VM queue to determine
the order in which virtual machines will be protected and which VSA proxies will back up each virtual
machine.

12

© 2023 Commvault.
Subclient Data Readers

The data readers setting in the advanced tab of the subclient defines the maximum number of
streams used for the backup. When the job starts, if there are more VMs than available streams, each
VM is allocated a single stream. If there are more streams than VMs, the coordinator automatically
instructs the data mover proxy to use multiple streams for the VM backups. Depending on the number
of available streams, each virtual disk in the VM is backed up as a single stream. This process is
dynamic so as a job progresses and more streams become available requiring fewer VMs to be
protected, multiple streams can be used to protect individual VMs.

13

© 2023 Commvault.
Datastore Distribution

If VMs within a subclient exist across multiple datastores, the coordinator assigns VMs to proxies with
one VM per datastore until the maximum stream count is reached. Each VM is assigned to a different
data mover proxy, balancing stream loads across proxies based on proxy resources. This distributes
the load across multiple datastores, improving backup performance and maintain a healthy datastore
state. In addition to the subclient Data Readers setting, a hard limit can be set for the maximum
number of concurrent VMs that can be protected within a single datastore using the
nVolumeActivityLimit additional setting.

14

© 2023 Commvault.
VSA Proxies

Commvault uses VSA proxies to facilitate the movement of virtual machine data during backup and
recovery operations. The VSA proxies are identified in the instance properties. For Microsoft Hyper-V,
each VSA proxy is designated to protect virtual machines hosted on the physical Hyper-V server. For
VMware, the VSA proxies are used as a pooled resource. This means that depending on resource
availability different proxies may be used to backup VSA subclients each time a job runs. This method
of backing up virtual machines provides higher scalability and resiliency.

VM and VSA Proxy Distribution Rules

Datastore distribution is the primary rule that determines the order in which VMs are backed up.
Additional rules that determine VM backup order are:
• Number of proxies available to back up a VM – The fewer proxies available, the higher the
VM is in the queue. This is also dependent on transport mode. If the transport mode is set to
Auto (default), SAN has highest priority, followed by HotAdd and then NDB mode. If a specific
transport mode is defined in the subclient, only proxies that can protect the VM can be used.
This can impact the available number of proxies and result in a higher queue priority.
• Number of virtual disks – VMs with more virtual disks are higher in the queue.
• Size of virtual machine – Larger VMs are higher in the queue.

15

© 2023 Commvault.
Stream Allocation and Proxy Throttling
During backup operations, the coordinator proxy gathers information on each data mover proxy to
determine the default maximum stream count each proxy can handle. This is based on the following:
• 10 streams per CPU
• 1 stream per 100MB available RAM
When the coordinator assigns jobs to the data mover proxies, it evenly distributes jobs until the
default maximum number of streams on a proxy is reached. Once the threshold is reached it no
longer assigns additional jobs to the proxy. If all proxies are handling the maximum number of
streams and there are still streams available, the coordinator assigns additional jobs to proxies using
a round-robin method. Throttling can be hard set on a per-proxy basis using registry keys.
• nStreamsPerCPU – limits the number of streams per CPU on the proxy
• nMemoryMBPerStream – sets the required memory on the proxy for each stream
• nStreamLimit – sets a limit on the total number of streams for a proxy
• bHardStreamLimit – sets a hard stream limit across all proxies within the VSA instance

16

© 2023 Commvault.
Virtual Machine Swap File Filtering

When backing up VMware or Hyper-V virtual machines, by default, the VSA filters the Windows page
file or Linux swap file. To achieve this, the system maps the virtual machine disk blocks from which
the page file or swap file is made. These blocks are skipped during the backups, significantly reducing
the storage footprint and the backup time.
It is possible to disable the skipping of page and swap files by creating the bSkipPageFileExtent
additional setting on the VSA proxy and by setting its value to 0 (zero).

Virtual Machine Filtering

Virtual machines can be filtered by browsing for VMs or adding specific criteria for VM filtering. This
can be useful when content is being defined at a parent level, but specific virtual machines are to be
excluded from backup. For instance, if the subclient is configured to auto-discover and protect all VMs
within a specific Datastore, but there are few virtual machines that do not require protection, they can
be added as filters. Virtual machines can be defined as filters at the subclient or at the backup set
level.
If your subclient’s content is defined using auto-discovery rules, define VM filters at the backup set
level to ensure that none of the subclients back up the VM.

17

© 2023 Commvault.
Virtual Disk Filtering
For some hypervisors, such as VMware and Hyper-V, disk level filtering can also be applied. This
provides the ability to filter disks based on host, datastore, VMDK, VHD or VHDX name pattern, or
hard disk number. This can be useful when certain disks do not require protection or if Commvault
agents installed within the VM are used to protect data.

Example: A database server requires protection. For shorter recovery points and more granular
backup and recovery functionality, a database agent can be used to protect application database
and log files. For system drives, the virtual server agent can be used for quick backup and
recovery. Disks containing the database and logs should be filtered from the VSA subclient. The
VSA will protect system drives and the application database agent will be used to protect
database daily and log files every 15 minutes. This solution provides shorter recovery points by
conducting frequent log backups, application aware backup and restores, and protects system
drives using the virtual server agent.

VSA Instance Configuration

Once the VSA software has been installed on all the desired proxies, the VSA pseudo client, or
instance, can be configured. When configuring the instance, a list of proxies must be defined. The first
proxy in the list acts as the VSA proxy coordinator.

Default Subclient Content

Right-click the default subclient > Properties > Content tab
The default subclient content tab contains a backslash entry like the Windows File System agents to
signify the subclient as a catch all. Any VMs not protected in other subclients are automatically
protected by the default subclient. Default subclient contents should not be changed, activity should
not be disabled, and the default subclient should be scheduled to backup regularly even if there are
no VMs in the subclient.
To avoid protecting VMs that do not need to be backed up, use the backup set level filters and add all
VMs that do not require protection. Complying with these best practices ensures that if a VM is added
in the virtualization environment, even if the Commvault system administrator is unaware of the VM, it
gets protected by the default subclient.

VM Content Tab
Right-click the desired subclient > Properties > Content tab
VSA subclient contents are defined using the Browse or Add buttons. Browse provides a vCenter-like
tree structure where resources can be selected at different levels including cluster and datastore.
Most environments should select subclient contents at the cluster level. For smaller environments, or
for optimal performance, defining subclient contents at the datastore level can be used to distribute
the load across multiple datastores.
The Add option is used to define discovery rules for VM content definition. Multiple rules can be
nested such as all Windows VMs in a specific datastore.

18

© 2023 Commvault.
VSA Advanced Restore Options

The VSA agent offers multiple live recovery features. Not all VSA features are supported on all
hypervisors.

Live Mount
Expand Client Computer Groups > VSA instance > Right-click the desired VM > All Tasks >
Live Mount
Live Mount allows you to run a virtual machine directly from a stored backup. Use this feature to
validate that backups are usable for a disaster recovery scenario, validate the content on the backup,
testing purposes, or access data from the virtual machine directly instead of restoring guest files.
Virtual machines that are live mounted are intended for short term usage and should not be used for
production. Changes to live mounted VMs or their data are not retained when the virtual machine
expires. The VM expiration period is set through a Virtual Machine policy.
When a live mount is initiated, an ESX server is selected to host the virtual machine, based on the
criteria set in the live mount virtual machine policy. The backup is exposed to the ESX server as a
temporary datastore. The configuration file for the live mounted VM is updated to reflect the name of
the new VM, disks are redirected to the datastore, and network connections are cleared and
reconfigured to the network selected for the live mounted VM. When this reconfiguration is complete,
the VM is powered on.

Tip: Using Live Mount for update validation

You are about to apply updates to a critical system and are concerned about the impacts on the
system.
Solution: Use Live Mount to power on the same system from the backups. Isolate it on its own
network to avoid duplicate hostname and IP address. Install and validate the update.

19

© 2023 Commvault.
Live File Recovery
Right-click the desired subclient or backup set > All Tasks > Browse and Restore > Virtual
Server tab
Live File Recovery provides expanded file system support, including ext4, and enables live browse of
backup data without requiring granular metadata collection during backups. This option supports
restores of files and folders from backups of Windows VMs and of UNIX VMs that use ext2, ext3,
ext4, XFS, JFS, or Btrfs file systems.
Live File Recovery can also be used to reduce backup times. This is a trade-off; using this feature
reduces backup time but increases the time required to browse files and folders. It is only supported
for backups to disk storage targets.
To recover files or folders from a backup, you can enable backup data to be mounted as a temporary
NFS datastore that can be used to browse and restore file and folders. The process is similar to an
ISO file that you right-click and mount on a Windows computer. The operating system virtually mounts
the ISO file and cracks it open to display the content. In the case of Live File Recovery, the Windows
MediaAgent locates the virtual machine's blocks in the disk library. These blocks are presented to the
Windows operating system through a virtual mount driver. The VM file system is then cracked open,
and the content is displayed in the console.
For Linux virtual machine, the file system cannot be mounted by the Windows MediaAgent. It requires
a virtual Linux MediaAgent on which the File Recovery Enabler for Linux (FREL) component must be
installed.
Enabling or disabling the Live File Recovery method is achieved by the Collect File Details backup
option of a subclient. If this is selected, traditional file recovery is used. If it is not selected, Live File
Recovery is used. The default for a new backup or schedule is to use Live File Recovery.
If Collect File Details was enabled, but you still want to use Live File Recovery, configure the following
additional setting key on the VSA proxy:
nEnforceLivebrowse with a value of 1
Performing a Live File Recovery is achieved through the usual guest files and folders recovery
screens. The difference is in the system mechanics.

Live VM Recovery
Right-click the desired subclient or backup set > All Tasks > Browse and Restore > Virtual
Server tab
The Live Recovery feature enables virtual machines (VMs) to be recovered and powered on from a
backup without waiting for a full restore of the VM. This feature can be used to recover a VM that has
failed and needs to be placed back in production quickly, or to validate that a backup can be used in a
disaster recovery scenario.
The disk library is presented to the virtualization environment. Then the VM is powered on from the
disk library. While it runs, the VM get moved back into the production Datastore using a storage
vMotion operation. All these tasks are accomplished automatically by Commvault.

20

© 2023 Commvault.
LiveSync
The LiveSync feature enables incremental replication from a backup of a virtual machine (source VM)
to a synchronized copy of the virtual machine (destination VM). The LiveSync operation opens the
destination VM and applies changes from the source VM backups since the last sync point. LiveSync
is not a real-time synchronization.
The LiveSync feature can initiate replication automatically after backups or on a scheduled basis (for
example, daily or once a week), without requiring any additional action from users. Using backup data
for replications minimizes the impact on the production workload by avoiding the need to read the
source VM again for replication. Additionally, in cases where corruption on the source VM is
replicated to the destination VM, users can still recover a point-in-time version of the source VM from
older backups. If no new backups have run since the last LiveSync, the scheduled LiveSync does not
run.
Use an incremental forever strategy with LiveSync. Run a first full backup that gets replicated to the
destination. Then only run incremental backups to apply the smallest changes possible to the
destination. Periodically, such as once a week, run a synthetic DASH full backup to consolidate
backups in a new full backup, without impacting the replication. If you execute a real full backup, the
entire machine must replicate to the destination.

LiveSync Configuration
Right-click the desired subclient or backup set > LiveSync > Configuration
Before you configure LiveSync, configure the vCenter client in the CommCell Console. If the
destination uses a different vCenter server, it must also be defined as a vCenter client. Then run the
initial VM backups. The VM must be backed up once and can then be added to a LiveSync schedule.

LiveSync from a Secondary Copy

Right-click the desired subclient or backup set > LiveSync > Configuration > Advanced >
Copy Precedence tab
By default, LiveSync replicates from backups in the primary copy of a plan. It is possible to modify this
behavior to restore from a secondary copy. This can be useful when the VM is backed up to a disk
library that is replicated to a remote site where the replicated machine resides.
When LiveSync is configured to use an auxiliary copy or backup copy, the LiveSync operation uses
the copy as the source rather than the primary backup. If the After Backup Job Completes option is
selected in the schedule, LiveSync automatically waits until the data is ready on the secondary copy
before running the LiveSync job.

LiveSync Monitor
Right-click the desired subclient or backup set > LiveSync > Monitor
The LiveSync Monitor tool is used to monitor and control LiveSync replication. In addition to the
replication status of VMs, replication can be enabled/disabled and VM failover/failback can be
initiated.

LiveSync Failover
From the LiveSync Monitor > Right-click the desired VM > Failover
From the LiveSync Monitor, the failover of a virtual machine can be initiated. It can be defined as a
planned failover for testing, or unplanned as in a disaster situation. Once a VM was failed over, a
failback operation can be executed. In a failback, the VM from the failover location is backed up and
synced back to the primary site.

21

© 2023 Commvault.
Prerequisites to use failover feature:
• The workflow engine must be installed on the CommServe server.
• The allowToolsToExecuteOnServerOrClient additional settings key with a value of 7 must be
created on the CommServe server.
• The VMs must have been synced at least once.

The Failover of a VM:

• Test Boot VM – Powers on the replicated VM. It is useful to test and ensure that it can be
used in the case of a disaster. The destination VM is not modified to avoid any conflicts with
the production VM.
• Planned Failover – The planned failover is useful to test the complete failover scenario or to
conduct maintenance on the primary site. A planned failover achieves the following tasks:
o Powers off the source VMs and performs an incremental backup of the source VMs
o Runs LiveSync to synchronize the destination VMs with the latest changes
o Disables LiveSync
o Powers on the destination VMs with the appropriate network connections and IP
addresses
• Unplanned Failover – The unplanned failover is used in a real disaster scenario where the
primary site is unavailable. The unplanned failover does not care about the primary site and
achieves the following tasks:
o Disable LiveSync
o Powers on the destination VMs with the appropriate network connections and IP
addresses

22

© 2023 Commvault.