XtremIO X2 – Deep Dive

XtremIO X2 Design Goals

Take the next step in the Inherit the best of Turn actionable insights
journey we envisioned XtremIO and make from our install base into
when creating XtremIO everything better customer value

3 © Copyright 2018 Dell Inc.

 What’s New with XtremIO?

Bigger Better Faster Cheaper Smarter

Up to

4X 25%
Better data reduction
80%
Better application latency
50% 2X
better rack density More copies
Lower $/GB

Native Replication

XtremIO X2
Next Generation
XtremIO Hardware Improved Management
and Software

                 Internal Use - Confidential

4
Why XtremIO?
 Why XtremIO?

Unmatched Efficiency
Maximize Efficiency with Deduplication and Compression.

App Integrated Copies

Rich Application Integration. No Compromise Copy services.

Consistent Performance
Inline, all the time data services with no performance impact

6 © Copyright 2017 Dell Inc.

A day in the life of a virtual data center

Thousands of VMs =
I/O blender
Provisioning &
Cloning new VMs
VMotion Operations
What Problem Are You Solving?

Creating a
simpler, faster, &
more agile virtual
data center?
 Experts Agree on Scale-Out
"The bottom line is that scale-out architectures with rich metadata support,
consistent latency, advanced snapshot technologies and advanced quality of
service are a prerequisite for fourth generation all-flash storage arrays.“

David Floyer

"The most successful vendors will be those that can make a smooth transition
from the traditional, dedicated application model to mixed workload
consolidation”
Eric Burgener

“Phase 3 starts to really require a scale-out system that can provide consistent
high performance and low latency”
George Crump

9 © Copyright 2017 Dell Inc.

Hardware Overview

10 © Copyright 2017 Dell Inc.

 X2 Under The Hood
Active Controller A Active Controller B
50%
More Cores 12
Cores
12
Cores
12
Cores
12
Cores
(Up to 384)

2x 1TB Infiniband 1TB

RAM RAM

More RAM

SAS 3.0
SAS 3.0
2 x 16 Gbps Fibre Channel 2 x 16 Gbps Fibre Channel
(Up to 16TB) 2 x 10 Gbps iSCSI 2 x 10 Gbps iSCSI

16Gb
Fibre Channel
Up to 72 SSDs (400GB or 1.92TB)

11 © Copyright 2018 Dell Inc.

 How Does It Scale?
X-Brick 1
SHARED MEMORY METADATA

Unique User Data

SHARED MEMORY METADATA

SHARED MEMORY METADATA SHARED MEMORY METADATA

RDMA
Unique User Data Unique User Data
FABRIC
SHARED MEMORY METADATA SHARED MEMORY METADATA

X-Brick 3 X-Brick 4
SHARED MEMORY METADATA

Unique User Data

SHARED MEMORY METADATA

X-Brick 2
12 © Copyright 2017 Dell Inc.
 Scale Capacity
Scale-out a cluster to
Scale-up X-Brick to 8 X-Bricks** of

138TB raw 1.1 PB raw

SSD SSD SSD SSD SSD SSD SSD SSD SSD

FLASH FLASH FLASH FLASH FLASH FLASH FLASH FLASH FLASH

SSD SSD SSD SSD SSD SSD SSD SSD SSD Up to

FLASH FLASH FLASH FLASH FLASH FLASH FLASH FLASH FLASH

SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH 5.5PB
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
SSD
FLASH
of effective capacity*
Active Controller Active Controller Active Controller
Active Controller Active Controller Active Controller

….........

*6:1 Data reduction

13 © Copyright 2017 Dell Inc.
** >4 X-Bricks and adding X-Bricks in field supported post-GA
 Scale Performance 1.7M MIXED IOPS
3.5M READ IOPS
<1MS Latency

220K MIXED IOPS

440K READ IOPS
<1MS Latency

SCALE-OUT <1MS LATENCY. RICH DATA SERVICES. NO TUNING.

14 © Copyright 2017 Dell Inc.

“FUNCTIONAL IOPS” VS. HERO NUMBERS

Measured end-to-end at the host

100% random
All flash pages non-zero
Array filled to 80% – forcing overwrites
ALL DATA
Any size LUNs, any LBA range
Mixed read/write I/O SERVICES
Increasing thread counts
Extended time periods ACTIVE!
 Highest Levels of Enterprise-Class Availability
100% Hardware Redundancy System-Wide Software Redundancy

Controllers

System-wide Data integrity checking w/ fault

DAE management module detection & advanced healing

 Active/Active controllers  2x Vault NVRAM

 2 x DAE controllers &  2x InfiniBand connect
connections  2x Port connectivity Secure distributed Advanced thermal Non-disruptive
 2x Power supplies/cables  2xEvery component journaling monitoring, alerting upgrades
& shutdown

NO SINGLE POINT OF FAILURE

16 © Copyright 2018 Dell Inc.
 New X-Brick Configurations

Configuration X-Brick X-Brick Cluster Size in X-

Minimum Raw Maximum Raw Bricks
X2-S 7.2 TB 28.8 TB Up to 4
X2- T 34.5 TB 69.1 TB 1**
X2-R 34.5 TB 138.2 TB*** Up to 8*

* Note: 8 X-Bricks configurations (up to 4 X-Bricks/cluster available now) and scale out in existing clusters coming in a future release
** Upgrade to X2-R (beyond 36 SSD’s and 1 X-Brick) at a price premium.
17 *** 3.84TB
© Copyright Drives Increase the maximum Raw Capacity
2018 Dell Inc.
 Storage Controller – NV-RAM

18 Dell - Internal Use - Confidential

Architecture Matters

19 © Copyright 2017 Dell Inc.

 XtremIO’s Unique Architecture
Secret sauce in software
Dual-Stage
Metadata Engine
Content Based DATA

Data Placement

100% In-Memory RAM

XtremIO Data
Protection

20 © Copyright 2017 Dell Inc.

 XTREMIO CONTENT
Content-aware technology ADDRESSIN and
deduplicatesG
INCOMING DATA STREAM

AB45CB7
F3AFBA3
CA38C90

220147A8
00325F7A
9963FE7B

1134F871

963FE7B
C

IO processing creates random
addressing that inherently load
balances
A
9
Finger printing technique ensures
that only unique block are
written to the flash media

21 © Copyright 2017 Dell Inc.

 XTREMIO CONTENT
Content-aware technologyADDRESSING

INCOMING DATA STREAM

AB45CB7
F3AFBA3
CA38C90

220147A8
00325F7A
1134F871
963FE7B

963FE7B
C

A
9

9
0 1 2 3 4 5 6 7 8 9 A B C D E
F

0 4 8 C
X-Brick #1 X-Brick #2 2 6 A E
Controller 1 Controller 1

1 5 9 D Controller 2 Controller 2 3 7 B F

22 © Copyright 2017 Dell Inc.

 Content addressing inherently load balances

Duplicate!
INCOMING DATA STREAM Do Not Write.

AB45CB7
CA38C90

F3AFBA3
Point to

20147A8
0325F7A
963FE7B

134F871

963FE7B
Previous

0 4 8 C
X-Brick #1 X-Brick #2 2 6 A E
Controller 1 Controller 1

1 5 9 D Controller 2 Controller 2 3 7 B F

23 © Copyright 2017 Dell Inc.

 XtremIO Architecture Benefits
• Extends flash endurance
• Always maximum performance
• No setup or tuning
• Foundation for advanced array data
services

24 © Copyright 2017 Dell Inc.

 Data Services
25 © Copyright 2017 Dell Inc.
 XtremIO In-Line, All-The-Time Data Services
XtremIO
Thin Provisioning Compression D@RE Native Replication
XtremIO XtremIO
Deduplication Data Protection Virtual Copies

All volumes Inline – Inline – Single Async.

Super
thin; block compressed “RAID Always-on efficient – Native
optimized written blocks only model” Encryption in-memory
for data once double with no perf Replication
meta data
saving No post- parity w/ impact
No post- copy 30s RPO
process 84% usable
process

26 © Copyright 2017 Dell Inc.

Write Boost

27 © Copyright 2017 Dell Inc.

 Write-Boost

• Leverages the NVRAM in the System

• Mirrors Incoming Writes for Higher Availability
• Host Ack is faster than X1
• Host Blocks <16K and Unaligned
– All the same performance

28 Dell - Internal Use - Confidential

 Performance Improvements

• Faster CPUs, more Cores, faster SAS/IB/FC

• IO Flow changes – “Write Boost”
– Particularly improves performance of smaller block writes

4k 100% Write X1 X2

Latency Improved BW Improved

By Write Boost By Aggregation

29 Dell - Internal Use - Confidential

 Let’s Compare….

30 Dell - Internal Use - Confidential

 Let’s Compare….

31 Dell - Internal Use - Confidential

 Let’s Compare….

32 Dell - Internal Use - Confidential

 Thin Provisioning

33 © Copyright 2017 Dell Inc.

 Thin Provisioning
Optimal Allocation—No Waste, Ever

 Always thin provisioned

 Perfect 16KB allocation
granularity
 No fragmentation or reclamation
penalty

34 © Copyright 2017 Dell Inc.

 XtremIO Content Engine – Zero Blocks
• Zero Blocks are identified
and will be ignored. Zero Blocks
• Unknown Read Blocks
are handled as Zero Blocks.
• No Metadata need to be allocated.
• No phys. SSD Blocks are written.
• No SSD Formating with Zero Blocks.
• Performance and Efficiency increases.
http://www.xtremio.com/saved-by-zero
35 © Copyright 2017 Dell Inc.
 How You Report “Zero” Writes Matters
• If you just count zero’s as
Non-Zero Data Reduction = 4:1 dedup’d blocks (like Pure
does):
– This example is 1TB:200GB =
Thin 1TB
5:1 Reported Reduction
Provisioned
2TB
“Zero”
“Zero” Writes
Writes 200GB • If you ignore zeros
Non-Zero
Non-Zero Writes
Writes 800GB
DDP+Compressed
DDP+Compressed 200GB (XtremIO):
Logical Volume Flash Media – 800GB:200GB = 4:1 Reported
Reduction

36 © Copyright 2017 Dell Inc.

 Is Reporting “Zero’s” as Dedup’d Wrong?
• Treating a “Zero” block like any other means tracking
metadata for that block (that contains nothing)
• This results in metadata bloat:
– Needlessly using memory, table space, overhead
– Slowing metadata by having to track/parse/index un-needed entries
• Treating “nothing” as a regular block is inefficient but has the
side benefit of padding reduction reporting

37 © Copyright 2017 Dell Inc.

Inline Compression

38 © Copyright 2017 Dell Inc.

 Compression - Algorithm

• Industry Uses LHA-LZH

– Not CPU Efficient
• XtremIO uses CDERS Algorithm
– Developed By EMC
– 2x CPU Efficient
– ~5% Lower Compression

39 Dell - Internal Use - Confidential

 Some History – X1

• X1 used an 8K Block Size

• Compressed 8K to one of 3 buckets :
– 8K (1:1)
– 4K (2:1)

0K-2K

2K-4K

4K-8K
– 2K (4:1)
• A block that compressed to 4097 bytes would be stored
uncompressed (8K)

40 Dell - Internal Use - Confidential

 Compression – X2 Buckets

• X2 uses a 16K block size

• Number of buckets increased to 15!
• Far more granular (4097 becomes 5k, not 8k)
• Individual blocks can compress up to 16:1
0K-1K

1K-2K

2K-3K

3K-4K

4K-5K

5K-6K

6K-7K

7K-8K

8K-9K

9K-10K

10K-11K

11K-12K

12K-13K

13K-15K

15K-16K
41 Dell - Internal Use - Confidential
 Compression - GUI

42 Dell - Internal Use - Confidential

XDP

43 © Copyright 2017 Dell Inc.

XtremIO Data Protection (XDP)
• No configuration

• No dedicated hot spares

• SSDs may fail in place

• Adapts to failures

• Built for Flash Media

44 Dell - Internal Use - Confidential

XDP – Low Overhead Algorithm

XDP RAID 1 RAID 5 RAID 6

Capacity Overhead 6-12% 50% 25% 20% SUPERIOR

USABLE CAPACITY

Write I/O Overhead

1.22
2 2 3 UP TO 4X
(stripe update) (64%) (64%) (146%) BETTER ENDURANCE

Read I/O Overhead

1.22 0
2 3 UP TO 4X
(stripe update) (64%) (146%) BETTER PERFORMANCE

45 Dell - Internal Use - Confidential

XDP Rebuilds & Hot Space
Allows SSDs To Fail-In-Place 3
4 5 failed SSDs
5 3
4
~330K IOPS

 Rapid, distributed,
content-aware rebuilds
 No performance impact 5 3
44
after rebuild completes
for up to 5 failed SSDs
per DPG

46 Dell - Internal Use - Confidential

XDP Rebuilds & Hot Space
Allows SSDs To Fail-In-Place 3
4 5 failed SSDs
5 3
4
~330K IOPS

 Rapid, distributed,
content-aware rebuilds
 No performance impact 5 3
44
after rebuild completes for
up to 5 failed SSDs per
DPG

© Copyright 2013 EMC Corporation. All rights reserved. 47

 SSD Failures

• SSD used have 2.5 million hour MTBF (300 years!)

• No dedicated hot spare disk
– 1 disk worth of space is reserved per DPG – “Hot spare space”
– Further rebuilds will use free space on array if available
• Can rebuild up to 4 failed disks per DPG (2 with 18 disks)
– 2 simultaneously
– Plus handle 2 additional failures (running off parity)
• Data is rebuilt, not entire drive

48 Dell - Internal Use - Confidential

Encryption

49 © Copyright 2017 Dell Inc.

 XtremIO Security – D@RE
DRIVE-BASED ENCRYPTION FOR MAXIMUM PROTECTION

• Data at Rest Encryption (D@RE)

• Self-Encrypting SSD (SED)

• Unique Data Encryption Key per drive (DEK)

• Zero performance impact on array

• All XtremIO advanced data services supported

 Federal mandate
Protects against
 Industry compliance
DRIVE LOSS
 Often eliminates drive shredding

50 © Copyright 2017 Dell Inc.

 XtremIO Virtual Copies (XVC)
And Integrated Copy Data Management

51 © Copyright 2017 Dell Inc.

 Enter XtremIO agile copy services
Unique in-memory copy technology

Performance

2X
Read, write, and latency same as volume

Space Efficient
No metadata bloat, no space
reservation, no moving blocks
Virtual copies
On X2 Platform
In-memory
Instant creation, immediate deletion,
flexible topography

Automated Workflows
App-integration with refresh, scheduling,
and self-service
52 © Copyright 2017 Dell Inc.
COPY DATA MANAGEMENT APPROACHES
…AND THE NEW INTEGRATED CDM OPTION

Full Subsets 3rd Party XtremIO

Traditional
Copies CDM In-Memory
Snapshots
Tools Integrated
“Clones” “Copy/Redirect- CDM
on-Write”

v
v
v v

COSTS $$$$ $ $$ $$$ FREE

PERFORMANCE No
Current CDM options force trade-offs
Compromise
between Cost,
Limited Performance,
Moderate Limited Agility,
Moderate Limitless
Array-Based
AGILITY
and Scalability
CDM
Limited Limited Limited Very Limited Massive
SCALABILITY

© Copyright 2015 EMC Corporation. All rights reserved. 53

 Traditional redirect-on-write snapshots

Production Snapshot

METADATA Full Metadata

Copy

DATA ON DISK

54 © Copyright 2017 Dell Inc.

 XVC: Metadata management is efficient
ALLOWING FOR A LARGE NUMBER OF CONCURRENT VIRTUAL COPIES

Production XVC

METADATA

Production MD Shared MD Snapshot MD

DATA ON DISK

55 © Copyright 2017 Dell Inc.

 XVC: Metadata management is efficient
NO DUPLICATION OF METADATA

Modify Data B @ Block 1

Production XVC

METADATA

Production MD Shared MD Snapshot MD

DATA ON DISK

56 © Copyright 2017 Dell Inc.

 XVC Writing to copies is same as writing to physical DB
Add Data B @
Block 5

Production XVC

METADATA

Production MD Shared MD Snapshot MD

DATA ON DISK

XTREMIO COPIES: INSTANT, EFFICIENT, ELEGANT

57 © Copyright 2017 Dell Inc.
 XVC vs.
XVC vs. TRADITIONAL SNAPSHOTS
traditional snapshots
XtremIO Vs. Vendor “A” DB Volumes, Steady State, IOPS Over Time Consider removing

70K IOPS XtremIO

• No impact on
IOPs

copy creation
• Consistent
performance on
prod and copy
XtremIO Vs. Vendor “A” DB Volumes, Steady State, Latency Over Time Vendor A
• IOPs drop by
50% to 35K
Latency

• Spikes to 20ms
1ms latency latency

58 © Copyright 2017 Dell Inc.

 X2 Solves the CDM Problem
iCDM - INSTANT REFRESH, ANY DIRECTION
More Copies?

App
Self DEVELOPMENT PRODUCTION
Service

App Integration &

orchestration ANALYTICS OPS+TEST

XtremIO Virtual Copies (XVC)

Consistent Multi-Dimensional Scale Why Not?

Performance & Data Services
59 © Copyright 2017 Dell Inc.
 iCDM is Popular : > 1.5M XVC’s At Work
A MAJORITY OF XTREMIO VIRTUAL
COPIES ARE WRITEABLE!
Read-only
46% Writeable
copies used for 54% copies used
protection, gold
for test/dev
master etc.

Source: XtremIO XVC type in Entire Install Base

60 © Copyright 2018 Dell Inc.

 XtremIO Virtual Copies Are Busy !
XTREMIO VIRTUAL COPIES HANDLE
~40% OF IO WORKLOAD
59% 60%

41% 40%
Total Read IOs Total Write IOs

IOs to Snapshots IOs to Volumes

XtremIO Virtual Copies v/s Volumes: IOs from Entire Install Base

61 © Copyright 2018 Dell Inc.

Meta-Data Aware
Native Replication
62 © Copyright 2017 Dell Inc.
 XtremIO Metadata-Aware Native Replication

EFFICIENT CONSISTENT RELIABLE

Only sends unique data to minimize Minimal to no performance impact Fast recovery across sites and
WAN bandwidth requirements with scale out architecture consistently low RPOs

63 © Copyright 2018 Dell Inc.

 Metadata Aware: Efficiently Transferring Changes Only
XtremIO snapshots
Following
XtremIO will only sync
generates aare the
list of
First Only
time
differences Globally
initialization
uniqueis Snapshot is being
changes a(deltas)
created atbetween between
frequency
the latest The transferred
informationtois the
stored
target
the blockstransferred
performed
latest are
fromtransferred
an initial
snapshot
derived
transferredfromsnapshot
RPO settingand the in a form
side and stored at at
of a snapshot the
and over
the the
snapshot.
new wire
snapshot
newlyincreated
the CG snapshot the target
targetside
array

Delta Snapshot

New Snapshot

First Snapshot New Snapshot New Snapshot First Initialization

Snapshot

64 © Copyright 2018 Dell Inc.

 Global Dedupe = Global Savings
Site 1 Site 3

APP APP APP APP APP APP

OS OS OS OS OS OS

Target Site 5
A B C D E A B C D E

99% WAN 38% Target

Savings ! Capacity
Site 2 Savings ! Site 4

A B C D E
APP APP APP APP APP APP
OS OS OS OS OS OS

A B C D E A B C D E

65 © Copyright 2018 Dell Inc.

 X2 Native Replication Robust Features
Flexible snap
retention management
Full GUI management and
monitoring
Configuration via simple XMS or CLI Wizards
REST API
Networking & Performance
IP V4 supported
VLANs supported
10GbE IP (onboard or Supported Operational Use Cases

iSCSI)
Up to 500 snaps per volume Replication
Testing
Suspend /
Terminate
Failover Failback Repurpose
Snapshot

w/ min 30 sec. RPO

66 © Copyright 2018 Dell Inc.
Scale-Out and Up

67 © Copyright 2017 Dell Inc.

 Scale-Out

• Adding Storage Capacity by adding additional X-Bricks

– Adds Capacity and Performance
• Requires balancing of data between X-Bricks

• AVAILABLE As of 6.2!

68 Dell - Internal Use - Confidential

 Scale-Up

• Adding Storage Capacity

– Without Adding X-Bricks/Storage Controllers
– New / Existing DPG within existing DAE
• No Data Rebalance Within DPG
– Near-Instant Operation

• AVAILABLE NOW!

69 Dell - Internal Use - Confidential

70
• DAE Layout

Dell - Internal Use - Confidential

• Next DPG... (+18)
• First Scale-Up (+6)

• Third Scale-Up (+6)

• Second Scale-Up (+6)
Scale Up Granularity

• Minimal Configuration (18)

Disk54
Slot #54 Disk36
Slot #36 Disk18
Slot #18 Disk00
Slot #00
Disk55
Slot #55 Disk37
Slot #37 Disk19
Slot #19 Disk01
Slot #01
Disk56
Slot #56 Disk38
Slot #38 Disk20
Slot #20 Slot #02
Disk02
Disk57
Slot #57 Disk39
Slot #39 Disk21
Slot #21 Slot #03
Disk03
Disk58
Slot #58 Disk40
Slot #40 Disk22
Slot #22 Disk04
Slot #04
Disk59
Slot #59 Disk41
Slot #41 Disk23
Slot #23 Disk05
Slot #05
SIM #0

Slot #60
Disk60 Disk42
Slot #42 Slot #24
Disk24 Disk06
Slot #06
Slot #61
Disk61 Disk43
Slot #43 Slot #25
Disk25 Disk07
Slot #07
Slot #62
Disk62 Disk44
Slot #44 Slot #26
Disk26 Slot #08
Disk08
iSIM #07 iSIM #05 iSIM #03 iSIM #01
iSIM #08 iSIM #06 iSIM #04 iSIM #02
Slot #63
Disk63 Disk45
Slot #45 Slot #27
Disk27 Disk09
Slot #09
DAE BACK

Slot #64 Disk46

Slot #46 Slot #28
Disk28 Disk10
Slot #10
DAE FRONT

Disk64
Slot #65
Disk65 Disk47
Slot #47 Slot #29
Disk29 Disk11
Slot #11
Slot #66
Disk66 Disk48
Slot #48 Slot #30
Disk30 Slot #12
Disk12
Slot #67
Disk67 Disk49
Slot #49 Slot #31
Disk31 Slot #13
Disk13
SIM #1

Slot #68
Disk68 Disk50
Slot #50 Slot #32
Disk32 Disk14
Slot #14
Slot #69
Disk69 Disk51
Slot #51 Slot #33
Disk33 Disk15
Slot #15
Slot #70
Disk70 Disk52
Slot #52 Slot #34
Disk34 Disk16
Slot #16
Slot #71
Disk71 Disk53
Slot #53 Slot #35
Disk35 Slot #17
Disk17

DPG-2 DPG-1
New Simple
HTML5 UI
71 © Copyright 2017 Dell Inc.
 Simple, Smart Management

Access
• No more Java
• Faster and better user
experience

Simple and
intuitive UI
• Easy drill-down & navigation
• Intelligent reports
• 1-2-3 Provisioning

72 © Copyright 2017 Dell Inc.

 Provisioning 1-2-3: Many-to-many mappings

  
Create volumes Create initiator groups Map volumes

73 © Copyright 2017 Dell Inc.

 New reports

Weekly peaks Latency heat map Block distribution

74 © Copyright 2017 Dell Inc.

 Volume Level Data Reduction Reporting
CAPACITY STATISTICS PER VOLUME/APPLICATION

• DRR: Weighted Data reduction

ratio per object
DRR
• Unique physical space: Unique
Volume Unique Physical Space blocks in the cluster that can be
reclaimed if the volume is deleted

Volume Snapshot Group • Copy Efficiency: shows the

savings from XtremIO unique
DRR
XVC technology – calculated on
Volume Protection Unique Physical Space Volume Snapshot Group level
copy
Copy Efficiency
Repurpose
copy

75 © Copyright 2018 Dell Inc.

DRR Per Volume

76 © Copyright 2017 Dell Inc.