You are on page 1of 67

Data Domain Advanced

Features and Functions


Velocity Partner Accreditation

EMC CONFIDENTIALINTERNAL USE ONLY.

Module Objectives
Upon completion of this module, you will be able to:
Describe VTL and VTL library planning
Describe snapshots, fastcopy, and data retention
Describe data replication and recovery

Describe DD Boost and integration with EMC


NetWorker
Describe capacity and throughput planning

Describe Data Domain system monitoring tools

EMC CONFIDENTIALINTERNAL USE ONLY.

Lesson: Virtual Tape Library (VTL)


Upon completion of this lesson, you will be able to:
Describe a Data Domain VTL
Describe VTL library planning

EMC CONFIDENTIALINTERNAL USE ONLY.

VTL Definition

Backup

Cache

Retention/
restore/
cloning

Disaster recovery/
archive/
offsite storage

Application/
media backup
server

Why should you use a VTL?

EMC CONFIDENTIALINTERNAL USE ONLY.

Configuration Terms
Barcode

Unique ID assigned to virtual tape when you create it


In the Data Domain OS aka: label, tape label

CAP

Cartridge access port (CAP), emulated tape enter/eject point for


moving tape to/from a library
In the Data Domain OS aka: mail slot

Library

Emulates physical tape library with tape drives, changer, CAPs, and
slots (cartridge slots)
In the Data Domain OS aka: autoloader, tape silo, tape mount, tape
jukebox, vault

Pool

Collection of tapes that map to a directory on a file system, used to


replicate tapes to a destination

Tapes

Represented in a system as files. You can export/import from a vault


to a library, move within a library across drives, slots, and CAPs
In the Data Domain OS aka: cartridge

Vault

Unused tapes stored in vault, tapes are in library/vault

EMC CONFIDENTIALINTERNAL USE ONLY.

VTL Library Planning


Up to 1,00,000
tape cartridges
in virtual vault
Fibre channel
256 virtual tape drives
(DD880 only)

128 virtual LTO-1, LTO-2,


LTO-3 tape drives
(all other models)

64 virtual libraries
20,000 slots per library
100 CAPs per library
1000 CAPs per system
800 GiB per tape

Robot loads\changes
tape cartridge

VTL

EMC CONFIDENTIALINTERNAL USE ONLY.

Capacity Planning
More planning needed at installation
Expired tapes NOT deleted, space not reclaimed till
tape is overwritten/deleted
Always create more slots that you need
Load tapes when you need them
Stop loading tapes once retention requirements are
met

EMC CONFIDENTIALINTERNAL USE ONLY.

Lesson: Fastcopy, Snapshots, and Data


Retention
Upon completion of this lesson, you will be able to:
Describe Data Domain fastcopy
Describe Data Domain snapshots
Describe Data Domain data retention

Explain the Data Domain cleaning process

EMC CONFIDENTIALINTERNAL USE ONLY.

Fastcopy

Source
directory

Copy

Target
directory

If you change source or target directory


while copying, they will not be equal.

EMC CONFIDENTIALINTERNAL USE ONLY.

Snapshots
Original copy

/data/ coll /backup

/data/ coll /backup/files

Snapshot taken at 22:24 GMT

EMC CONFIDENTIALINTERNAL USE ONLY.

Snapshot copy

/data/ coll /backup / .snapshot


/data/ coll /backup/files/ .snapshot

22:24 GMT snapshot saved

10

Retention Lock
Prevents retention-locked files from being deleted/modified for up to 70 years

Archive software
or user initiates

Licensed feature

Retention locked files can be stored, encrypted, and replicated

EMC CONFIDENTIALINTERNAL USE ONLY.

11

Retention Lock Flow

1. License/enable retention lock


3. Create file

2. Set min/max retention period

4. Lock file (set retention period)


- Extend retention-locked file
- Delete expired retention-locked file
5. Transfer file to Data Domain system

EMC CONFIDENTIALINTERNAL USE ONLY.

12

Configure Client File Retention Period

current time

1. User creates file and sets last access time (atime) to desired retention period
Client must initiate retention lock

minimum retention period

valid atime period


maximum retention period

2. Data Domain system administrator sets min/max retention periods on Data Domain system

3. File either committed as a retention-locked file or ignored

EMC CONFIDENTIALINTERNAL USE ONLY.

13

File System Cleaning


Cleaning reclaims physical storage occupied by deleted objects

SW backups to Data Domain

File A deleted with no retention lock

File A deleted at next cleaning

File B deleted, retention lock initiated

File B maintained until retention lock period ends

EMC CONFIDENTIALINTERNAL USE ONLY.

14

Cleaning
What?

Container 2

Reclaim space

Container 1

Why?

House keeping (reclaim dead segments)

Performance (rewrite duplicate data)

Container 1

Disk block
House keeping

or
Performance tuning

Disk blocks

Container 3

dead

copy Forward

Container 2

valid

Free space

EMC CONFIDENTIALINTERNAL USE ONLY.

Free space

Disk block

15

Lesson: Replication and Recovery


Upon completion of this lesson, you will be able to:
Describe the types of Data Domain replication
Identify how replication improves storage
Describe the data recovery process

EMC CONFIDENTIALINTERNAL USE ONLY.

16

Data Replication

LAN

Source

WAN

Destination

New deduplicated compressed data is automatically replicated to destination

EMC CONFIDENTIALINTERNAL USE ONLY.

17

Data Domain Replication Types


Collection: for entire site backup
Directory: for partial site backup

Pool: for VTL files/tape backup

EMC CONFIDENTIALINTERNAL USE ONLY.

18

Data Domain Collection Replication

/backup

Source

Recovers
entire system
/backup

Destination
Immediate accessibility
Read only
User accounts/passwords
replicated from source
Works with encrypted files
Works with retention lock

EMC CONFIDENTIALINTERNAL USE ONLY.

19

Data Domain Directory Replication

/backup/dir a

/backup/dir a

Destination must have


available storage

CIFS and NFS clients ok


Do not mix CIFS/NFS
data in same directory

/backup/dir b
Destination

Source

Recovers selected data

Destination directory
created automatically

/backup/dir b

Works with encryption

Works with retention


lock
Destination

EMC CONFIDENTIALINTERNAL USE ONLY.

20

Data Domain Pool Replication

pool 1

pool 1
pool 3

pool 3
pool 2

pool 2
Source

EMC CONFIDENTIALINTERNAL USE ONLY.

Destination

Works like directory replication

Destination doesnt require VTL license

21

Replication Context

WAN
Source

EMC CONFIDENTIALINTERNAL USE ONLY.

Destination

22

Replication Topologies
Destination

Source

Destination

Source

1 to 1

Bi-directional

Source

Destination

Destination

Source

1 to many

Many to 1
Source

Primary
source/
Destination

Cascaded

EMC CONFIDENTIALINTERNAL USE ONLY.

Source
Destination

Primary
source/
destination

Destination

Cascaded 1-to-many

23

Recover Data
In case of disaster,
recover off-site replica

WAN

Clients

File server

Backup
server

On site

Replication

Off site
disaster recovery

You can configure a Data Domain system to store


backup data and retain onsite for 30-90 days

EMC CONFIDENTIALINTERNAL USE ONLY.

24

Why Resynchronize Recovered Data?

Recreate deleted context

WAN

Out of space

Resynchronization

Source

Destination
Convert collection to
directory replication

EMC CONFIDENTIALINTERNAL USE ONLY.

25

Lesson: Data Domain Boost


Upon completion of this lesson, you will be able to:
Describe DD Boost
Describe replica awareness
Describe how DD Boost works with EMC NetWorker

Describe supported network topologies


Describe DD Boost advanced load balancing and link
failover feature

EMC CONFIDENTIALINTERNAL USE ONLY.

26

DD Boost
Provides standard/centralized management features
through backup software
Works with industry standard backup software
EMC Networker
Symantec NetBackup (Data Domain plug-in required)
Symantec Backup Exec (Data Domain plug-in required)

Enables advanced load balancing and failover


Requires licenses on Data Domain System

DD Boost
Replication (if used) Note: Your backup software might require
license to enable the feature. Verify your backup software
documents.

EMC CONFIDENTIALINTERNAL USE ONLY.

27

DD Boost (contd.)
Clients send
data to backup
server

Clients

Manages connections
between backup applications
and Data Domain systems with DD Boost

LAN

OST
plug-in
LAN

Optimized
protocol
for high
throughput

DD
Boost

Backup Server
Deduplication/compression
occur in backup server

EMC CONFIDENTIALINTERNAL USE ONLY.

Less data sent


over LAN

Deduped Data
Stored

28

Replica Awareness
Initiates and tracks
replication for easy
management and
disaster recovery
You manage replication from
backup server console
OST
plug-in
DD
Boost
Backup site

DD
Boost

WAN
replication
Disaster recovery site

Backup
Server

Archive to tape as needed

EMC CONFIDENTIALINTERNAL USE ONLY.

29

DD Boost Advantage
Without DD Boost

With DD Boost

Backup server(s) not aware of


Data Domain replica(s)
Recovery is manual process

Backup server dedupes data


and minimizes network
bandwidth use
Replication and recovery are
centrally configured and
monitored

Optimized
deduplication

DD Boost server
With DD Boost
OST
plug-in

Manually
configured replication
Backup

Replication

Replication
engine

Backup
server

DD Boost server

Without OST

EMC CONFIDENTIALINTERNAL USE ONLY.

30

NetWorker Work Flow


Control
Data

NetWorker
Server

Local
Data Domain system

Save Set 1
update
control data

Remote
Data Domain system

New data backup


(Save Set 1)
1

Save
Set 1

Clone 1

7
Clone 1
update
control
data

EMC CONFIDENTIALINTERNAL USE ONLY.

Done (Save Set 1)


4

Save
Set 1

5
Start clone
(Clone 1)

Done (Clone 1)

Data
transfer

Clone 1

31

Lesson: Capacity and Throughput Planning


Upon completion of this lesson, you will be able to:
Describe capacity planning and its importance
Describe throughput planning and its importance

EMC CONFIDENTIALINTERNAL USE ONLY.

32

Monitor File System Space Use


Factors that effect how fast data on disk grows
Size of data sets getting backed up
Compressability of data getting backed up
Retention period specified in backup software

Monitor disk use closely when you back up large data


sets that show low compression factors and have
large retention times
You can get more accurate space-use view from CLI
Use filesys show space to monitor post-compression
data growth

EMC CONFIDENTIALINTERNAL USE ONLY.

33

Space Graph
Compression Ratio:
Pre-compression/ Data Collection

Available Space on DDS

Amount of Data within


Backup Application

EMC CONFIDENTIALINTERNAL USE ONLY.

Cumulative physical data


written to DDS

34

Space Graph (contd.)

What does the saw-tooth line for compression ratio represent?

EMC CONFIDENTIALINTERNAL USE ONLY.

35

Compression Factor Calculation


Compression factor

Original bytes
Data Domain system data written

What does cleaning do to this equation?


It decreases the Data Domain system data written (denominator)
and thus increases the compression factor.

EMC CONFIDENTIALINTERNAL USE ONLY.

36

Capacity Planning: Determine Capacity


Needs
How much?
Data size (TB)
Data type
Full backup size
Compression rate
(deduplication)

EMC CONFIDENTIALINTERNAL USE ONLY.

How long?
Retention policy
(duration)
Schedule

Capacity
needs

37

Determine Capacity Needs (contd.)


Data Domain system internal indexes and other
components use variable storage amounts
depending on data type and file sizes
If different data sets are sent to identical systems,
one system may, over time, have room for
more/less backup data than another
Challenging data types
Pre-compressed (multimedia, .zip, and .tiff)
Encrypted

EMC CONFIDENTIALINTERNAL USE ONLY.

38

Compression Requirements with Variables


5x Nearline and archive

Incremental + weekly full backup with two weeks retention


Daily full backup with one week retention
Nearline and archival use compression tends to be capped here

10x Overall compression

Incremental + weekly full backup with one month of retention


Daily full backup with two-three weeks retention

20x Overall compression

Incremental + weekly full backup with two-three months retention


Daily full backup with three-four weeks retention

EMC CONFIDENTIALINTERNAL USE ONLY.

39

Calculate Required Capacity


Total space required
Required
capacity
1st full backup

EMC CONFIDENTIALINTERNAL USE ONLY.

Incremental
backup 4

Weekly full
backup
number of
weeks

Required
capacity

40

Calculate Required Throughput

Largest backup
Required throughput
Backup time window

Example

EMC CONFIDENTIALINTERNAL USE ONLY.

6 TB
10 hrs

600 GB/hr

41

System Model Capacity and Performance


Maximum capacity is amount of usable data storage
space
Maximum capacity based on max number of drives
supported by a model
Maximum throughput is achieved using either VTL
interface and 4Gbps Fibre Channel or DD Boost and
10Gb Ethernet
Current model throughput and capacity specifications
http://www.datadomain.com/products/

EMC CONFIDENTIALINTERNAL USE ONLY.

42

Select Model
Required capacity
100

Capacity %

Maximum logical capacity

Required throughput
100
Maximum throughput

Throughput %

Be conservative when determining which model to use


Use 75-85% of model capacity and throughput
(factor 15-25% buffer for capacity and throughput)

EMC CONFIDENTIALINTERNAL USE ONLY.

43

Calculate Capacity Buffer for Selected


Models
Required capacity
Maximum capacity

100%

% of Maximum capacity

DD140 example
840 GB
860 GB

100%

97%

3% Buffer not ok

51%

51% Buffer ok

DD610 example
840 GB
1650 GB

EMC CONFIDENTIALINTERNAL USE ONLY.

100%

44

Match Required Capacity to Model


Specifications
For example

Required capacity = 840 GB


Ensure capacity buffer is big enough
DD140

DD610

OR?
860 GB

EMC CONFIDENTIALINTERNAL USE ONLY.

1,650 GB with 7 drives

45

Calculate Performance Buffer for Selected


Models
Required throughput
Maximum throughput

100%

% of Maximum throughput

100%

89% 11% Buffer not ok

100%

53%

DD610 example
600 GB/hr
675 GB/hr
DD630 example
600 GB/hr
1126 GB/hr

EMC CONFIDENTIALINTERNAL USE ONLY.

47% Buffer ok

46

Match Required Capacity to Model


Specifications
For example

Required throughput = 600 GB/hr


Ensure performance buffer is big enough
DD 610

DD630

OR?
860 GB/hr

EMC CONFIDENTIALINTERNAL USE ONLY.

1.1 GB/hr

47

Lesson: System Monitoring Tools


Upon completion of this lesson, you will be able to:
Describe Data Domain system monitoring tools

SNMP
syslog
autosupport
SUP

EMC CONFIDENTIALINTERNAL USE ONLY.

48

Monitoring a Data Domain System


2. syslog

1. SNMP

Daily
alerts and
autosupport reports

3. autosupport

4. SUB

Daily
alerts and
autosupport reports

Data Domain
system administrator

Data Domain
technical support

Alert

EMC CONFIDENTIALINTERNAL USE ONLY.

49

SNMP
You can monitor a Data Domain system via SNMP utilities
You can integrate the Data Domain Management
Information Base (MIB) into SNMP monitoring

EMC CONFIDENTIALINTERNAL USE ONLY.

50

Syslog (Remote Logging)

Sends system messages to remote syslog server


System messages
Port 514

LAN

Port 514

syslog server

Uses TCP port 514

You collect logs

EMC CONFIDENTIALINTERNAL USE ONLY.

51

Autosupport
Easy to install just once at system setup
Helps solve/prevent system problems
Provides timely notification of significant issues
Enables rapid response time to address or prevent problems
Includes critical system data to aid support case triage and
management

EMC CONFIDENTIALINTERNAL USE ONLY.

52

Autosupport System
via SMTP

autosupport@
autosupport.datadomain.com

Summary autosupport report


Detailed autosupport report

System history

Daily alert summary

Other vendors

Reboots
Warnings
Integration to
other systems
Reports

Data Domain technical support


(support case)

EMC CONFIDENTIALINTERNAL USE ONLY.

53

Autosupport Types
autosupport
types

Non scheduled

Scheduled

Detailed
autosupport
report
sent 6 am

EMC CONFIDENTIALINTERNAL USE ONLY.

Daily alert
summary
email
sent 8 am

Alerts

Reboots

warning
failure

54

Autosupport Via Enterprise Manager


Data Domain systems provide alerts,
autosupport reports, and logs
Access through Enterprise Manager

EMC CONFIDENTIALINTERNAL USE ONLY.

55

Autosupport Reports

Subscribers
receive daily
detailed
reports

EMC CONFIDENTIALINTERNAL USE ONLY.

Using SMTP, sent to Data Domain


technical support daily at 6 am
local time (default)
Contains system ID, uptime
information, system command
outputs, runtime parameters, logs,
system settings, status and performance
data, and other debugging information
Long text report (500-800K)
Sections parsed into data warehouse
for analysis and reporting

56

Detailed Autosupport Report Contents

EMC CONFIDENTIALINTERNAL USE ONLY.

57

Daily Summary Autosupport


Sent daily at 8
am

Provides
summary
autosupport
report

Tells you if the


system is ok or
not. If not, no
email receipt is
received

EMC CONFIDENTIALINTERNAL USE ONLY.

Uses
autosupport
email
distribution list
This is an example
of an alert

58

Alerts

Unique numerical ID
Alert severity

EMC CONFIDENTIALINTERNAL USE ONLY.

Date and time alert occurred

Subsystem where
alert occurred

Physical component where alert


occurred

59

Alerts Notification
Creates support case
Sent immediately upon detection

Terse description of event


Has separate email distribution list

EMC CONFIDENTIALINTERNAL USE ONLY.

60

Logs

Every Sunday at 3 am
1. New log file opened
2. Old log file renamed

CLI: log view filename

EMC CONFIDENTIALINTERNAL USE ONLY.

61

Support Upload Bundle (SUB)


Large (multi-GB sized) tar file
Contains
OS settings and log files
System files (not customer data files) identified as needed
for system diagnosis by Data Domain support and
engineering
Used to triage and diagnose a Data Domain system in the field
CLI commands used to generate and optionally send (via http)
SUB to Data Domain support site
Generated by sysadmin on Data Domain system via GUI/CLI

EMC CONFIDENTIALINTERNAL USE ONLY.

62

Module Summary
Key points covered in this module include:
VTL and VTL library planning
Snapshots, fastcopy, and data retention
Data replication and recovery

DD Boost and integration with EMC NetWorker


Capacity and throughput planning
Data Domain system monitoring tools

EMC CONFIDENTIALINTERNAL USE ONLY.

63

Product Demo
Click the link below to view a demonstration on Data
Domain.

Launch Demo

EMC CONFIDENTIALINTERNAL USE ONLY.

64

EMC CONFIDENTIALINTERNAL USE ONLY.

65

PROPERTIES
On passing, 'Finish' button:
On failing, 'Finish' button:
Allow user to leave quiz:
User may view slides after quiz:
User may attempt quiz:

Goes to Next Slide


Goes to Next Slide
At any time
At any time
Unlimited times

THANK YOU

EMC CONFIDENTIALINTERNAL USE ONLY.

67