XenServer

Storage Integration Deep Dive

Agenda

XenServer 5.5 Storage Architecture

Multipathing

Vendor Integration

StorageLink

© 2009 Citrix Systems, Inc. — All rights reserved 2

 Citrix XenServer & Essentials 5.5 Family
Stage Management NEW

Lab Management
Platinum
Provisioning Services (p+v)
Edition

Essentials Provisioning Services (virtual)

Workload Balancing
NEW
StorageLink TM
Enterprise
Edition High Availability
Performance Monitoring
Workflow Studio Orchestration

Live Migration (XenMotion) Active Directory Integration

NEW
64bit, Windows, Linux Workloads Generic Storage Snapshotting
XenServer Free XenCenter Management
Edition Shared Storage (iSCSI, FC, NFS)
No socket restriction

© 2009 Citrix Systems, Inc. — All rights reserved 3

XenServer 5.5
Storage Architecture
Expanded Backup Support

© 2009 Citrix Systems, Inc. — All rights reserved 5

Storage Technologies
XenServer 5.0 / 5.5 XenServer 5.0 XenServer 5.5
NFS / EXT3 iSCSI / FC iSCSI / FC

Storage Repository Storage Repository Storage Repository

Filesystem LUN LUN

VHD header VHD header

.VHD .VHD LVM LVM
file file Logical Logical LVM LVM
Volume Volume Logical Logical
Volume Volume

LVM Volume Group LVM Volume Group

VM virtual disk

© 2009 Citrix Systems, Inc. — All rights reserved 6

LVM XenServer 5.5 (LVHD)
• Replaces LVM for SRs
• Hosts VHD files directly on LVM volumes
• Best of both worlds
• Features of VHD
• Performance of LVM
• Adds Advanced Storage features
• Fast Cloning
• Snapshots
• Fast and simple upgrade
• Backwards compatible
© 2009 Citrix Systems, Inc. — All rights reserved 7
Multipathing
Why using Multipathing?

• Path redundancy to storage

• Performance increase by load sharing algorithms
• Many fibre channel environments by default have
multiple paths
LUN
1

FC LUN Storage
1
controller 1
HBA 1
LUN LUN
FC 1
Storage 1
HBA 2 LUN
controller 2
1

XenServer FC Switches Storage Subsystem

© 2009 Citrix Systems, Inc. — All rights reserved 11
Enabling Multipathing

• xe host-param-set other-
config:multipathing=true
uuid=host_uuid
• xe host-param-set other-
config:multipathhandle=dmp
uuid=host_uuid
• Note: Do not enable
multipathing by other ways
(e.g. CLI)!!!

© 2009 Citrix Systems, Inc. — All rights reserved 12

 XenServer supports 2 multipathing
technologies
Device Mapper Multipathing (DMP) RDAC MPP (mppVhba)

Default yes no

XenServer version >= 5.0 >= 5.0 Update 2

4.1 CLI
Management by XenCenter yes no

Support wide storage range only LSI controller based storage

Driver / Daemon multipathd mppVhba driver
CLI path check multipath –ll mpputil
Configuration /etc/multipath-enabled.conf /etc/mpp.conf
(requires execution of
/opt/xensource/bin/update-initrd)

See details: http://support.citrix.com/article/ctx118791

© 2009 Citrix Systems, Inc. — All rights reserved 13
DMP vs RDAC MPP

• Check if RDAC MPP is running

• lsmod | grep mppVhba
• „multipath –ll“ would show MD device as output (if DMP is active)

• Use only 1 technology

• When RDAP MPP is running use it
• Otherwise use DMP

© 2009 Citrix Systems, Inc. — All rights reserved 14

MPP RDAC: Path check
Lun #0 - WWN: 600a0b80001fdf0800001d9c49b0caa1
• mpputil ----------------
LunObject: present
DevState: OPTIMAL

Controller 'A' Path

--------------------
Path #1: LunPathDevice: present
DevState: OPTIMAL

Path #2: LunPathDevice: present

DevState: OPTIMAL

Controller 'B' Path

--------------------
Path #1: LunPathDevice: present
DevState: OPTIMAL

Path #2: LunPathDevice: present

DevState: OPTIMAL

© 2009 Citrix Systems, Inc. — All rights reserved 15

DMP: Path check

• Monitoring using XenCenter

• Monitoring using CLI
• Command: multipath -ll

© 2009 Citrix Systems, Inc. — All rights reserved 16

Multipathing & Software iSCSI
iSCSI with Software Initiator

• IP addressing to be done by XenServer Dom-0

• Multipathing also to be done by XenServer Dom-0
• Dom-0 IP configuration is essential
Storage
XenServer LAN Ctrl 1
LUN
Dom-0 1
LUN
1 IP IP LUN
IP
1

NIC 1 LUN
1

XenServer Switches Storage Subsystem

© 2009 Citrix Systems, Inc. — All rights reserved 18
Best practice configuration:
iSCSI storage with multipathing
Subnet:
• Separation of subnets also on IP base 255.255.255.0

Subnet 1

Port 1 IP:
192.168.1.201 Storage
LAN
NIC 1 IP: Port 2 IP: Adapter 1
NIC 1 192.168.1.10 192.168.2.201

NIC 2 IP:
NIC 2 192.168.2.10
Port 1 IP:
192.168.1.202 Storage LUN
LAN 1
Port 2 IP: Adapter 2
192.168.2.202

XenServer Subnet 2 FC Switches Storage Subsystem

© 2009 Citrix Systems, Inc. — All rights reserved 19

Not recommended configurations for
multipathing and iSCSI:

Both server NICs in same subnet Mixing of NIC teaming and multipathing

Subnet 1 Subnet 1

NIC 1 IP: Team IP:

NIC 1 192.168.1.10 NIC 1 192.168.1.10

NIC 2 IP:
NIC 2 192.168.1.11 NIC 2

XenServer XenServer

© 2009 Citrix Systems, Inc. — All rights reserved 20

Multipathing with Software Initiator
XenServer 5

• XenServer 5 supports Multipathing with iSCSI

software initiator
• Prerequisites are:
• iSCSI target uses same IQN on all ports
• iSCSI target ports operate in portal mode

• Multipathing reliability has been enhanced

massively in XenServer 5.5

© 2009 Citrix Systems, Inc. — All rights reserved 21

How to check if iSCSI target operates in
portal mode?

• Execute
iscsiadm -m discovery --type sendtargets --portal <ip
address of 1 target>

• Output must show alls IPs of the

target ports with identical IQN
• Example:
192.168.0.161:3260,1 iqn.strawberry:litchie
192.168.0.204:3260,2 iqn.strawberry:litchie

• When connecting to iSCSI target

using XenCenter Storage
Repository Wizard, also all target
IPs should show up after Discovery

© 2009 Citrix Systems, Inc. — All rights reserved 22

NetApp Integration
NetApp Storage

• NetApp Storage supports Multipathing

• For configuring NetApp storage and modification of
multipath.conf see whitepaper
http://support.citrix.com/article/CTX118842
• NetApp typically supports portal mode for iSCSI
 multipathing for iSCSI SW Initiator is supported
• Especially for low-end NetApp storage (e.g. FAS2020) with
limited LAN adapters special considerations take place

© 2009 Citrix Systems, Inc. — All rights reserved 24

NetApp low-end storage (iSCSI)

• Often limited by NIC configuration

• Example: 2 NICs per head
• 1 aggregate / LUN is represented by 1 head at a time (other
head for fault tolerance)
• Thus: 2 NICs effectively can be used for storage connection
• Typically Filer delivers non-block-based protocols (e.g. CIFS)
which also require redundancy as well as block based
protocols (e.g. iSCSI)

© 2009 Citrix Systems, Inc. — All rights reserved 25

Example FAS2020: Scenario 1
no network reduncancy for iSCSI and CIFS
separation of networks

CIFS Network
Controller 0
(active)

NIC 0

NIC 1

NIC 0 Controller 1
(fault tolerance)

NIC 0
NIC 1
NIC 1

iSCSI Network

© 2009 Citrix Systems, Inc. — All rights reserved 26

Example FAS2020: Scenario 2
network redundancy for iSCSI and CIFS
no separation of networks

CIFS & iSCSI Network

Controller 0
(active)

NIC 0
vif /
NIC Bond
NIC 1 bond

NIC 0 Controller 1
(fault tolerance)

NIC 0
NIC 1 vif /
NIC 1 bond

© 2009 Citrix Systems, Inc. — All rights reserved 27

Example FAS2020: Scenario 3
network redundancy for iSCSI (multipathing) and CIFS
separation of networks
NIC Bond
CIFS VLAN Ctrl 1
NIC 0
CIFS VLAN iSCSI VLAN (active)

NIC 1
iSCSI VLAN NIC 0
Vif /
NIC 1 bond
Multipathing
Controller 1
NIC 2 (fault tolerance)
Same configuration
NIC 3

© 2009 Citrix Systems, Inc. — All rights reserved 28

Dell / Equalogic Integration
Dell Equalogic Support

• XenServer 5.5 includes Adapter

(min. firmware 4.0.1 required)
• Redundant path configuration does
not depend on using adapter or not
• All PS series are supported as
running same OS
• StorageLink Gateway support
planned

© 2009 Citrix Systems, Inc. — All rights reserved 30

Dell / Equalogic
• See whitepaper for Dell / Equalogic storage
http://support.citrix.com/article/CTX118841
• Each Equalogic has two controllers
• Only 1 controller is active
• Uses „Group ID“ address on storage side
(similar to bonding / teaming on server side)
• Only connection over group ID, no direct connection to the iSCSI ports
possible
• Therefore multipathing cannot be used  bonding on XenServer side

© 2009 Citrix Systems, Inc. — All rights reserved 31

Datacore Integration
Multipathing architecture with Datacore
Subnet:
• Different IQNs for targets – no portal mode possible!! 255.255.255.0

Subnet 1

Port 1 IP:
192.168.1.201 Storage
Controller
NIC 1 IP: 1
NIC 1 192.168.1.10 IQN 1

NIC 2 IP:
NIC 2 192.168.2.10
IQN 2 Storage LUN
Controller 1
Port 2 IP: 2
192.168.2.202

XenServer Subnet 2 Switches

© 2009 Citrix Systems, Inc. — All rights reserved 33

Datacore hints

• Special attention for software iSCSI

• Follow Datacore technical bulletin: TB15
ftp://support.datacore.com/psp/tech_bulletins/Tech
BulletinsAll/TB15b_Citrix%20XenServer_config_50
1.pdf
• Datacore in VM
• O.k. when not using HA
• Configuration possible, but take care about booting the whole environment

• Take care when updating XenServer

© 2009 Citrix Systems, Inc. — All rights reserved 34

StorageLink
Logical advancement of XenServer
integrated storage adapters

Netapp & Equalogic

Storage adapter

© 2009 Citrix Systems, Inc. — All rights reserved 36

Citrix StorageLink Overview (XenServer)
Data Path

XenServer Control Path

Guest

iSCSI / F
C

Snap-in
for XenServer

StorageLink

© 2009 Citrix Systems, Inc. — All rights reserved 37

Leveraging the best of virtualization and
storage

© 2009 Citrix Systems, Inc. — All rights reserved 38

 StorageLink Overview
XenServer Hyper-V
VSM Parent
Netapp EqualLogic DOM0 VDS
Bridge Partition

XenServer Hyper-V

Data Path Data Path

LUN LUN LUN
Virtual Storage Manager
Storage Repository

Control Path

SAN/NAS
LUN LUN LUN
LUN LUN LUN
Storage Repository
Storage Repository

© 2009 Citrix Systems, Inc. — All rights reserved 39

 StorageLink Gateway Overview
SMI-S is the
preferred Web Services Interface (SOAP/.NET)
method of
integration as Citrix StorageLink Gateway Core
it requires no Storage Modules Network Modules Hypervisor Modules
custom
Fabric
development SMIS Proxy NetApp
(SMIS)
Hyper-V XenServer
work
Dell
Dell

Vendor Vendor Vendor X Vendor Y

Vendor’s
SMI-S SMI-S Storage Storage
NetApp FAS980

SMI-S
NetApp FAS980

CIMOM CIMOM Adapter Adapter

a cti vi ty

status
powe r

provider activity
sta tus

po we r

NetApp

HP
St or ag eWor ks

UID
HP
Sto ra geW orks
MSA150 0cs
Vendor-specific VSM
Storage Adapters run in
U ID

H P Stora geWorks

Mod ular Smart Array 15 0 0

1 4 7 10

3 6 9 12

HP
Sto ra ge Work s

HP EVA series
M SA1 500c s
U ID

HP
Sto rage Work s
M SA1 500cs
UI D

HP Sto ra geWorks

separate processes
Mo dul ar Smart Array 15 00
HP Sto rag eWo rks

Mo dula r Sma rt Arr ay 15 00

© 2009 Citrix Systems, Inc. — All rights reserved 40

Storage Technologies
XenServer 5.5 XenServer 5.5
iSCSI / FC iSCSI / FC +
Storage Repository Storage Repository

LUN

VHD header VHD header

LVM LVM LUN LUN

Logical Logical
Volume Volume

LVM Volume Group

VM virtual disk

© 2009 Citrix Systems, Inc. — All rights reserved 41

Snapshot types

XenServer (free) Essentials Enterprise

Snapshot type Software based snapshot Hardware based snapshot
(also software based snapshot possible
when not using StorageLink)
LUN access model LVM (1 LUN=x times VDI) LUN-per-VDI (1 LUN = 1 VDI)

Performance Good Superior

Utilization On XenServer host On storage subsystem

Overhead Low lowest

© 2009 Citrix Systems, Inc. — All rights reserved 43

StorageLink: Microsoft look-and-feel

© 2009 Citrix Systems, Inc. — All rights reserved 44

Essentials: Example usage scenario
Effective creation of VMs from template
VM clone 1x VM clone 3x

VM 1. Copy of LUN 1. Copy of LUN

Template
2. Modification of Zoning 2. Modification of Zoning

3. Creation of VM 3. Creation of VM

4. Assignment of LUN to VM 4. Assignment of LUN to VM

1. Copy of LUN

2. Modification of Zoning

3. Creation of VM

4. Assignment of LUN to VM

1. Copy of LUN

2. Modification of Zoning Effectiveness:

3. Creation of VM • Fast-Cloning using Storage Snapshots
4. Assignment of LUN to VM • Fully automated storage and SAN configuration
LUN • for FC and iSCSI

© 2009 Citrix Systems, Inc. — All rights reserved 45

StorageLink: Supported Storages

• StorageLink HCL
http://hcl.vmd.citrix.com/SLG-HCLHome.aspx

© 2009 Citrix Systems, Inc. — All rights reserved 46