WHITE PAPER – DECEMBER 2017

CONNECTING IP-BASED
(NFS OR ISCSI)
EXTERNAL STORAGE TO
VMWARE CLOUD FOUNDATION™
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Contents
Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Overview of Cloud Foundation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Cloud Foundation Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Software iSCSI Support in vSphere 6.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Use Cases for Adding Secondary Network Attached Storage to a
Cloud Foundation Deployment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
iSCSI and NFS Storage Array. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Trunking Required VLANs Where Necessary to Enable Access to
Network Attached Storage Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Configuring Network Access for NFS or iSCSI Connectivity. . . . . . . . . . . . . . . . . . . 6

Use SDDC Manager to Create Additional Data Center
Networks (Automated) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Create VMkernel Interfaces on Each VMware Host (Manual). . . . . . . . . . . . . . . 8
Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Add Storage to VMware Cluster – NFS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Add Storage to VMware Cluster – iSCSI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

About the Author. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

W H I T E PA P E R | 2
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Audience
This document contains guidance for VMware Cloud Foundation™ administrators 
who want to add secondary network attached storage, either NFS or iSCSI, to their
Cloud Foundation deployment.

Overview of Cloud Foundation

Cloud Foundation is the VMware unified Software-Defined Data Center (SDDC)
platform for the private and public cloud. It brings together VMware vSphere®,
VMware vSAN™, and VMware NSX® in a natively integrated stack. It delivers
enterprise-ready cloud infrastructure with unique automation and management
capabilities for a consistent operational experience across clouds.

In Cloud Foundation, administrators use the SDDC Manager interface to allocate

available servers to a logical construct called a workload domain. When a workload
domain is created, servers are configured into a VMware cluster that is ready for use
by a VMware administrator and is optimized for various use cases such as virtual
infrastructure or virtual desktop infrastructure (VDI) deployments.

Cloud Foundation Architecture

In the prescriptive Cloud Foundation topology, server nodes connect to supported
top-of-rack (TOR) switches via copper twinax cables from host LAN on Motherboard
(LOM), Network Daughter Card (NDC), or PCIe-based 10GbE network adapters with
SFP+ ports.

Host connectivity in Cloud Foundation consists of two 10GbE connections that

support a VMware vSphere Distributed Switch™ (VDS). These uplinks are configured
using link aggregation group (LAG).

On each vSphere host, the two 10GbE host network interfaces are cabled as shown
in Figures 1 and 2.

W H I T E PA P E R | 3
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Customer
Network

DC Uplink DC Uplink

Cloud Foundation–Supported TOR Cloud Foundation–Supported TOR

MC-LAG

LAG (LACP)

Dell PowerEdge R630 with 10G SFP+

Data Path

Figure 1. Cloud Foundation Default Topology and Cabling (Single Rack)

Customer
Network

DC Uplink DC Uplink

Cloud Foundation–Supported Interrack Switches (Multirack)

Cloud Foundation–Supported TOR Rack 1 Cloud Foundation–Supported TOR Rack 1

MC-LAG

LAG (LACP)
Dell PowerEdge R630 with 10G SFP+
Data Path

Figure 2. Cloud Foundation Default Topology and Cabling (Multirack)

W H I T E PA P E R | 4
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Software iSCSI Support in vSphere 6.5

As of October 2017, VMware vSphere 6.5 supports iSCSI multipathing over LAG if
port binding is not used. See “Limitations of the LACP Support on a vSphere
Distributed Switch.” Prior to vSphere 6.5, using a software iSCSI initiator specified
that initiators and targets must be on the same IP subnet.

With vSphere 6.5, routed iSCSI is supported for connectivity between initiators and
targets in different IP subnets. This makes it much easier to configure multipathing
without requiring that the initiator and target be on the same network. See “Best
Practices for Configuring Networking with Software iSCSI.”

Further, vSphere 6.5 also supports overriding the default gateway to provide different
gateways for options such as VMware vSphere vMotion®, iSCSI, vSAN, and so on.
See “Overriding the Default Gateway of a VMkernel Adapter” and “Configure the
VMkernel Adapter Gateway by Using ESXCLI.”

This paper focuses on configuring a Cloud Foundation environment to attach to

secondary network attached storage—that is, NFS or iSCSI—that might exist in a
customer environment, connecting over the existing LAG configuration. The
prioritization of this guidance is simplicity, network availability, and performance.

Use Cases for Adding Secondary Network Attached Storage to a

Cloud Foundation Deployment
The following use cases are common for adding secondary network attached storage
to a Cloud Foundation deployment:
• Migrating existing virtual machine (VM) workloads to a Cloud Foundation
environment
• Leveraging existing storage array investments to run workloads in a
Cloud Foundation environment

Other use cases include storing content libraries and templates, storing backup
images, and more.

NOTE: Cloud Foundation includes native vSAN support. This document does not suggest
replacement of vSAN resources; it recommends the addition of existing network attached
storage for the previously listed use cases.

W H I T E PA P E R | 5
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Assumptions
iSCSI and NFS Storage Array
NFS storage arrays often export an NFS share via a single IP address. iSCSI arrays
often present storage resources via multiple iSCSI portal IP addresses that are
assigned or bound to one or more iSCSI target interfaces on the array. These portals
or targets might require access from an isolated L2 network segment or might
support routable iSCSI connectivity. iSCSI arrays might employ security by requiring
CHAP authentication or use network-based security that defines permitted iSCSI
initiator IP and subnet.

NOTE: The procedures for configuring CHAP or network-based security are out of scope for this
guidance document. For more information, see iSCSI array vendor documentation and existing
VMware software iSCSI documentation.

Trunking Required VLANs Where Necessary to Enable Access to

Network Attached Storage Resources
Any required VLANs for NFS or iSCSI connectivity should be trunked on the
upstream network switch ports serving the TOR uplinks. The procedures to configure
VLANs on northbound switches is out of scope for this guidance document. For
more information, see switch vendor documentation.

NOTE: VMware Cloud Foundation does not currently support automated disjoint L2
or L3 networking.

Configuring Network Access for

NFS or iSCSI Connectivity
In this guidance document, SDDC Manager is used to create new data center
networks. SDDC Manager automates network or VLAN configuration changes
in the TOR switches and enables assignment of a new port group or VLAN to
Cloud Foundation workload domains. When a data center network is assigned to a
workload domain, a new distributed port group is created in the workload domain
VDS and is configured with the new VLAN. After these networks have been created
or assigned, VMkernel interfaces must be created manually on each cluster host to
enable connectivity to storage.

Use SDDC Manager to Create Additional Data Center

Networks (Automated)
After the required VLANs have been trunked to the TOR uplinks, from the customer
network side, use SDDC Manager to create new data center connections, including
new VLANs and assigned IP subnet information. Then map these to the selected
workload domain(s).

NOTE: The Data Center Connection Wizard is located in SDDC Manager under Settings ➝
Network Settings ➝ Data Center.

W H I T E PA P E R | 6
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Figure 3. SDDC Manager – Create a New Data Center Connection

After a new data center connection has been created or saved, SDDC Manager
automates the configuration of TOR switch ports and VDS with new port groups
mapped to new VLANs.

Figure 4. SDDC Manager – Activate Automated Configuration of TOR Switch Ports and VDS

W H I T E PA P E R | 7
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Create VMkernel Interfaces on Each VMware Host (Manual)

Create one or more VMkernel interfaces on each host in a cluster that requires
connectivity to the iSCSI or NFS storage resources.

To add a VMkernel network interface, right-click a host, select Add Networking, and
then select VMkernel Network Adapter. Click Next.

Figure 5. SDDC Manager – Add a VMkernel Network Interface

Browse to select the target network. List the newly created data center network using
the following format:

vRack-DPortGroup-External-<Datacenter Network Connection Name>.

Figure 6. SDDC Manager – List the Data Center Network

When the Network name is displayed, click Next.

W H I T E PA P E R | 8
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

On the Port Properties page, leave all settings at default. Click Next.

Figure 7. SDDC Manager – Leave Default Port Properties

Configure IPv4 settings as needed. Click Next. Verify the summary and click Finish.

Figure 8. SDDC Manager – Configure IPv4 Settings

NOTE: The new VMkernel interface is defined with an MTU of 1500 bytes by default.

If an MTU of 9000 is preferred, set end-to-end network ports accordingly.

NOTE: The host-facing TOR ports are configured by default with an MTU of 9216. The TOR
uplink ports are not configured by default with an MTU of 9216.

W H I T E PA P E R | 9
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Troubleshooting
Host CLI commands—that is, esxcfg-vmknic and vmkping—can be used to
display configured interfaces and to ping network resources from specific
VMkernel interfaces.

Figure 9. SDDC Manager – Use Host CLI Commands to Display Configured Interfaces

Add Storage to VMware Cluster – NFS

To add an NFS datastore to a cluster of hosts, right-click the first host, choose
Storage, and select New Datastore.

Figure 10. SDDC Manager – Add an NFS Datastore to a Cluster

Select Type and click Next.

W H I T E PA P E R | 1 0
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Figure 11. SDDC Manager – Select Datastore Type

Continue with the wizard to add the NFS storage to the host.

NOTE: After storage has been added to the first host, use the shortcut to mount the datastore
on the remaining hosts in the cluster.

Right-click the newly added datastore. Choose Mount Datastore to Additional Hosts.

Figure 12. SDDC Manager – Mount Datastore to Additional Hosts

Select the additional hosts from the list and click OK.

W H I T E PA P E R | 1 1
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Figure 13. SDDC Manager – Select Additional Hosts

The datastore now displays all connected hosts.

Figure 14. SDDC Manager – View Connected Hosts

Add Storage to VMware Cluster – iSCSI

To add iSCSI storage devices, start by adding the software iSCSI adapter. On one of
the hosts in the cluster, navigate to Configure. Choose Storage Adapters. Click the
plus sign and choose Software iSCSI adapter.

W H I T E PA P E R | 1 2
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Figure 15. SDDC Manager – Add VMware Software iSCSI Adapter

Next, choose iSCSI Software Adapter. Click Targets. Under Dynamic Discovery,
click Add.

Figure 16. SDDC Manager – Use Dynamic Discovery to Add Targets

W H I T E PA P E R | 1 3
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

NOTE: Using Dynamic Discovery, an initiator transmits a Send Targets request to the iSCSI
server. Return all available targets to the initiator. Target rescanning reoccurs automatically if the
adapter is reset or if hosts reboot—that is, LCM updates.

Enter the iSCSI Server (portal) IP address and Port. Click OK.

Figure 17. SDDC Manager – Add Targets

Add any other required Targets.

Figure 18. SDDC Manager – Add All Targets

After the Targets have been entered, click iSCSI Software Adapter. Then click the icon
below it to scan the selected host storage adapter for new storage devices.

Figure 19. SDDC Manager – Scan Host Storage Adapter for New Storage Devices

W H I T E PA P E R | 1 4
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

Select Devices to display all discovered devices.

Figure 20. SDDC Manager – Display Discovered Devices

Click Paths to ensure that all paths have been discovered. In Figure 21, there are two
portal IPs registered, each bound to a unique target interface. Each target interface
presents four 10G LUNs.

Figure 21. SDDC Manager – See Registered Portal IPs

Repeat the process on each host in the cluster until all hosts can detect all devices
and paths.

W H I T E PA P E R | 1 5
 CONNECTING IP-BASED (NFS/ISCSI) EXTERNAL STORAGE
TO VMWARE CLOUD FOUNDATION

The esxcfg-mpath host command can facilitate determination of device paths and
availability status.

Figure 22. SDDC Manager – See Device Paths and Availability Status

Conclusion
VMware Cloud Foundation is an integrated cloud infrastructure platform
designed for the hybrid cloud. This paper provides guidance for configuring a
Cloud Foundation deployment to attach to existing NFS or iSCSI storage resources
in the customer network environment, using a simplified process that provides
availability and performance.

Acknowledgments
The author thanks Ven Immani, Todd Simmons, and Chandra Avutu from the
VMware Cloud Foundation Marketing, Architecture, and Engineering teams for
their expertise, guidance, and review of this document.

About the Author

Jason Shaw is a technical product manager within the Integrated Systems
Business Unit at VMware. His focus is on VMware Cloud Foundation.

References
“Limitations of the LACP Support on a vSphere Distributed Switch”

“Add a VMkernel Network Adapter in the VMware Host Client”

“Best Practices for Configuring Networking with Software iSCSI”

W H I T E PA P E R | 1 6
VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 www.vmware.com
Copyright © 2017 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at
http://www.vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. and its subsidiaries in the United States and other jurisdictions. All other marks and names mentioned herein
may be trademarks of their respective companies. Item No: VMW-VCF-ISCSI-USLET-101
12/17