ESXi Host Configuration

KNOWLEDGE - OFFLINE PDF COPY
ESXi Host Configuration

Generated On Thu, 09 Feb 2023 16:47:38 GMT
PUBLIC
For the latest information, please visit

http://support-sandbox.purestorage.com/Solutions/VMware_Platform_Guide/User_Guides_for_VMware_Solutions/
FlashArray_VMware_Best_Practices_User_Guide/dddVMware_ESXi_Host_Configuration
© 2015-2023 Pure Storage® (“Pure”), Portworx® and associated its trademarks can be found here as and its virtual patent marking
program can be found here. Third party names may be trademarks of their respective owners. The Pure Storage products and
programs described in this documentation are distributed under a license agreement restricting the use, copying, distribution,
and decompilation/reverse engineering of the products. No part of this documentation may be reproduced in any form by any means
without prior written authorization from Pure Storage, Inc. and its licensors, if any. Pure Storage may make improvements and/or
changes in the Pure Storage products and/or the programs described in this documentation at any time without notice. This
documentation is provided "as is" and all express or implied conditions, representations and warranties, including any implied
warranty of merchantability, fitness for a particular purpose, or non-infringement, are disclaimed, except to the extent that
such disclaimers are held to be legally invalid. Pure shall not be liable for incidental or consequential damages in connection
with the furnishing, performance, or use of this documentation. The information contained in this documentation is subject to
change without notice.
1
This section reviews the recommended configuration of ESXi hosts for use with the Pure Storage FlashArray.
VMware Native Multipathing Plugin (NMP) Configuration

VMware offers a Native Multipathing Plugin (NMP) layer in vSphere through Storage Array Type Plugins (SATP)
and Path Selection Policies (PSP) as part of the VMware APIs for Pluggable Storage Architecture (PSA). The
SATP has all the knowledge of the storage array to aggregate I/Os across multiple channels and has the
intelligence to send failover commands when a path has failed. The Path Selection Policy can be either “Fixed”,
“Most Recently Used” or “Round Robin”.
Round Robin Path Selection Policy

To best leverage the active-active nature of the front end of the FlashArray, Pure Storage requires that you
configure FlashArray volumes to use the Round Robin Path Selection Policy. The Round Robin PSP rotates
between all discovered paths for a given volume which allows ESXi (and therefore the virtual machines running
on the volume) to maximize the possible performance by using all available resources (HBAs, target ports, etc.).
BEST PRACTICE: Use the Round Robin Path Selection Policy for FlashArray volumes.
The I/O Operations Limit

The Round Robin Path Selection Policy allows for additional tuning of its path-switching behavior in the form of a
setting called the I/O Operations Limit. The I/O Operations Limit (sometimes called the “IOPS” value) dictates
how often ESXi switches logical paths for a given device. By default, when Round Robin is enabled on a device,
ESXi will switch to a new logical path every 1,000 I/Os. In other words, ESXi will choose a logical path, and start
issuing all I/Os for that device down that path. Once it has issued 1,000 I/Os for that device, down that path, it
will switch to a new logical path and so on.
Pure Storage recommends tuning this value down to the minimum of 1. This will cause ESXi to change logical
paths after every single I/O, instead of 1,000.
This recommendation is made for a few reasons:
1. Performance. Often the reason cited to change this value is performance. While this is true in certain
cases, the performance impact of changing this value is not usually profound (generally in the single digits
of a percentage performance increase). While changing this value from 1,000 to 1 can improve
performance, it generally will not solve a major performance problem. Regardless, changing this value can
improve performance in some use cases, especially with iSCSI.
2. Path Failover Time. It has been noted in testing that ESXi will fail logical paths much more quickly when this
https://support.purestorage.com/Solutions/VMware_Platform_Guide/User_Guides_for_VMware_Solutions/FlashArray_V…
Updated: Thu, 09 Feb 2023 16:47:38 GMT
©2023 Copyright Pure Storage. All rights reserved.
2
value is set to a the minimum of 1. During a physical failure of the storage environment (loss of a HBA,
switch, cable, port, controller) ESXi, after a certain period of time, will fail any logical path that relies on that
failed physical hardware and will discontinue attempting to use it for a given volume. This failure does not
always happen immediately. When the I/O Operations Limit is set to the default of 1,000 path failover time
can sometimes be in the 10s of seconds which can lead to noticeable disruption in performance during this
failure. When this value is set to the minimum of 1, path failover generally decreases to sub-ten seconds.
This greatly reduces the impact of a physical failure in the storage environment and provides greater
performance resiliency and reliability.
3. FlashArray Controller I/O Balance. When Purity is upgraded on a FlashArray, the following process is
observed (at a high level): upgrade Purity on one controller, reboot it, wait for it to come back up, upgrade
Purity on the other controller, reboot it and you’re done. Due to the reboots, twice during the process half
of the FlashArray front-end ports go away. Because of this, we want to ensure that all hosts are actively
using both controllers prior to upgrade. One method that is used to confirm this is to check the I/O balance
from each host across both controllers. When volumes are configured to use Most Recently Used, an
imbalance of 100% is usually observed (ESXi tends to select paths that lead to the same front end port for
all devices). This then means additional troubleshooting to make sure that host can survive a controller
reboot. When Round Robin is enabled with the default I/O Operations Limit, port imbalance is improved to
about 20-30% difference. When the I/O Operations Limit is set to 1, this imbalance is less than 1%. This
gives Pure Storage and the end user confidence that all hosts are properly using all available front-end
ports.
For these three above reasons, Pure Storage highly recommends altering the I/O Operations Limit to 1. For
additional information you can read the VMware KB regarding setting the IOPs Limit.
BEST PRACTICE: Change the Round Robin I/O Operations Limit from 1,000 to 1 for
FlashArray volumes on vSphere. This is a default configuration in all supported vSphere
releases.
To fully utilize CPU resources, set the host's active power policy to high performance.
ESXi Express Patch 5 or 6.5 Update 1 and later

Starting with ESXi 6.0 Express Patch 5 (build 5572656) and later (Release notes) and ESXi 6.5 Update 1 (build
5969303) and later (release notes), Round Robin and an I/O Operations limit is the default configuration for all
Pure Storage FlashArray devices (iSCSI and Fibre Channel) and no configuration is required.
A new default SATP rule, provided by VMware by default was specifically built for the FlashArray to Pure
Storage’s best practices. Inside of ESXi you will see a new system rule:
Name Device Vendor Model Driver Transport Options Rule Group Claim
Options Default PSP PSP Options Description
------------------- ------ -------- ---------------- ------ --------- -------------------------- ---------- -----------------------------------
----------- -------------- --------------------------------------------------------------------------
VMW_SATP_ALUA PURE FlashArray
system VMW_PSP_RR iops=1
For information, refer to this blog post:
3
https://www.codyhosterman.com/2017/0...e-now-default/
Configuring Round Robin and the I/O Operations Limit

If you are running earlier than ESXi 6.0 Express Patch 5 or 6.5 Update 1, there are a variety of ways to configure
Round Robin and the I/O Operations Limit. This can be set on a per-device basis and as every new volume is
added, these options can be set against that volume. This is not a particularly good option as one must do this
for every new volume, which can make it easy to forget, and must do it on every host for every volume. This
makes the chance of exposure to mistakes quite large.
The recommended option for configuring Round Robin and the correct I/O Operations Limit is to create a rule
that will cause any new FlashArray device that is added in the future to that host to automatically get the Round
Robin PSP and an I/O Operation Limit value of 1.
The following command creates a rule that achieves both of these for only Pure Storage FlashArray devices:
esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -V "PURE" -M "FlashArray" -P "VMW_PSP_RR" -O
"iops=1" -e "FlashArray SATP Rule"
This must be repeated for each ESXi host.
This can also be accomplished through PowerCLI. Once connected to a vCenter Server this script will iterate
through all of the hosts in that particular vCenter and create a default rule to set Round Robin for all Pure
Storage FlashArray devices with an I/O Operation Limit set to 1.
Connect-VIServer -Server <vCenter> -Credential (Get-Credential)

Get-VMhost | Get-EsxCli –V2 | % {$_.storage.nmp.satp.rule.add.Invoke(@{description='Pure Storage
FlashArray SATP';model='FlashArray';vendor='PURE';satp='VMW_SATP_ALUA';psp='VMW_PSP_RR';
pspoption='iops=1'})}
Furthermore, this can be configured using vSphere Host Profiles:
4
It is important to note that existing, previously presented devices will need to be manually set to Round Robin
and an I/O Operation Limit of 1. Optionally, the ESXi host can be rebooted so that it can inherit the multipathing
configuration set forth by the new rule.
For setting a new I/O Operation Limit on an existing device, see Appendix I: Per-Device NMP Configuration.
Note that I/O Operations of 1 is the default in 6.0 Patch 5 and later in the 6.0 code branch, 6.5
Update 1 and later in the 6.5 code branch, and all versions of 6.7 and later.
5
Enhanced Round Robin Load Balancing (Latency Based PSP)
With the release of vSphere 6.7 U1, there is now a sub-policy option for Round Robin that actively
monitors individual path performance. This new sub-policy is called "Enhanced Round Robin Load Balancing"
(also known as Latency Based Path Selection Policy (PSP)). Before this policy became available the ESXi host
would utilize all active paths by sending I/O requests down each path in a "fire and forget" type of fashion,
sending 1 I/O down each path before moving to the next. Often times this resulted in performance penalties
when individual paths became degraded and weren't functioning as optimally as other available paths. This
performance penalty was invoked because the ESXi host would continue using the non-optimal path due to
limited insight into the overall path health. This now changes with the Latency Based PSP by monitoring each
path for latency, along with outstanding I/Os, allowing the ESXi host to make smarter decisions on which paths
to use and which to exclude in a more dynamic manner.
How it Works
Like all other Native Multipathing Plugin (NMP) policies this sub-policy is set on a per LUN or per
datastore basis. Once enabled the NMP begins by assessing the first 16 user I/O requests per path and
calculates their average latency. Once all of the paths have been successfully analyzed the NMP will then
calculate the average latency of each path and use this information to determine which paths are healthy
(optimal) and which are unhealthy (non-optimal). If a path falls outside of the average latency it is deemed non-
optimal and will not be used until latency has reached an optimal response time once more.
After the initial assessment, the ESXi host then repeats the same process outlined above every 3 minutes. It will
test every active path, including any non-optimal paths, to confirm if the latency has improved, worsened, or
remained the same. Once again those results will be analyzed and used to determine which paths should
continue sending I/O requests and which should be paused to see if they report better health in the next 3
minutes. Throughout this process the NMP is also taking into account any outstanding I/Os for each path to
make more informed decisions.
Configuring Round Robin and the Latency Based Sub-Policy

If you are using ESXi 7.0 or later then no changes are required to enable this new sub-policy as it is the new
recommendation moving forward. In an effort to make things easier for end-users a new SATP rule has been
added that will automatically apply this rule to any Pure Storage LUNs presented to the ESXi host:
Name Device Vendor Model Driver Transport Options Rule Group Claim
Options Default PSP PSP Options Description
VMW_SATP_ALUA PURE FlashArray
system VMW_PSP_RR policy=latency
If your environment is using ESXi 6.7U1 or later and you wish to utilize this feature, which Pure Storage supports,
then the best way is to create a SATP rule on each ESXi host, which can be done as follows:
esxcli storage nmp satp rule add -s "VMW_SATP_ALUA" -V "PURE" -M "FlashArray" -P "VMW_PSP_RR" -O
"policy=latency" -e "FlashArray SATP Rule"
6
Alternatively, this can be done using PowerShell:
Connect-VIServer -Server <vCenter> -Credential (Get-Credential)

Get-VMhost | Get-EsxCli –V2 | % {$_.storage.nmp.satp.rule.add.Invoke(@{description='Pure Storage
FlashArray SATP';model='FlashArray';vendor='PURE';satp='VMW_SATP_ALUA';psp='VMW_PSP_RR';
pspoption='policy=latency'})}
Setting a new SATP rule will only change the policy for newly presented LUNs, it does not get
applied to LUNs that were present before the rule was set until the host is rebooted.
Lastly, if you would like to change an individual LUN (or set of LUNs) you can run the following command to
change the PSP to latency (where device is specific to your env):
esxcli storage nmp psp roundrobin deviceconfig set --type=latency --device=naa.

624a93708a75393becad4e43000540e8
Tuning
By default the RR latency policy is configured to send 16 user I/O requests down each path and evaluate each
path every three minutes (180000ms). Based on extensive testing, Pure Storage's recommendation is to leave
these options configured to their defaults and no changes are required.
BEST PRACTICE: Enhanced Round Robin Load Balancing is configured by default on ESXi 7.0
and later. No configuration changes are required.
Verifying Connectivity
It is important to verify proper connectivity prior to implementing production workloads on a host or volume.
This consists of a few steps:
1. Verifying proper multipathing settings in ESXi.

2. Verifying the proper numbers of paths.
3. Verifying I/O balance and redundancy on the FlashArray.
The Path Selection Policy and number of paths can be verified easily inside of the vSphere Web Client.
7
This will report the path selection policy and the number of logical paths. The number of logical paths will
depend on the number of HBAs, zoning and the number of ports cabled on the FlashArray.
The I/O Operations Limit cannot be checked from the vSphere Web Client—it can only be verified or altered via
command line utilities. The following command can check a particular device for the PSP and I/O Operations
Limit:
esxcli storage nmp device list -d naa.<device NAA>
Please remember that each of these settings is a per-host setting, so while a volume might be configured
properly on one host, it may not be correct on another.
Additionally, it is also possible to check multipathing from the FlashArray.
A CLI command exists to monitor I/O balance coming into the array:
purehost monitor --balance --interval <how long to sample> --repeat <how many iterations>
8
The command will report a few things:
1. The host name.

2. The individual initiators from the host. If they are logged into more than one FlashArray port, it will be
reported more than once. If an initiator is not logged in at all, it will not appear.
3. The port that the initiator is logged into.
4. The number of I/Os that came into that port from that initiator over the time period sampled.
5. The relative percentage of I/Os for that initiator as compared to the maximum.
The balance command will count the I/Os that came down from a particular initiator during the sampled time
period, and it will do that for all initiator/target relationships for that host. Whichever relationship/path has the
most I/Os will be designated as 100%. The rest of the paths will be then denoted as a percentage of that
number. So if a host has two paths, and the first path has 1,000 I/Os and the second path has 800, the first path
will be 100% and the second will be 80%.
A well balanced host should be within a few percentage points of each path. Anything more than 15% or so
might be worthy of investigation. Refer to this post for more information.
Please keep in mind that if the Latency Based PSP is in use that IO may not be 1 to 1 for all
paths to the Array from the ESXi hosts.
There is nothing inherently wrong with the IO not being balanced 1 to 1 for all paths as the
Latency Bases PSP will be distributing IO based on which path has the lowest latency. With
that said, a few percentage points difference shouldn't be cause for alarm, however if there are
paths with very little to no IO being sent down them this should be something investigated in
the SAN to find out why that path is performing poorly.
The GUI will also report on host connectivity in general, based on initiator logins.
9
This report should be listed as redundant for all hosts, meaning that it is connected to each controller. If this
reports something else, investigate zoning and/or host configuration to correct this.
For a detailed explanation of the various reported states, please refer to the FlashArray User Guide which can be
found directly in your GUI:
10
Disk.DiskMaxIOSize
The ESXi host setting, Disk.DiskMaxIOSize, controls the largest I/O size that ESXi will allow to be sent from ESXi
to an underlying storage device. By default this is 32 MB. If an I/O is larger than the Disk.DiskMaxIOSize value,
ESXi will split the I/O requests into segments under the configured limit.
If you are running an older release of ESXi (versions listed below) this setting needs to be modified if and only
if you are on an old version and have an environment running the following scenarios:
1. If a virtual machine is using EFI (Extensible Firmware Interface) instead of BIOS and is using VMware
Hardware Version 12 or earlier.
2. If your environment utilizes vSphere Replication.
3. If your environment contains VMs which house applications that are sending READ or WRITE requests
larger than 4 MB.
11
4. The environment is using Fibre Channel with one of the above scenarios (this issue is not present with
iSCSI).
VMware has resolved this issue in two places--fixing it in ESXi itself (ESXi now reads the maximum supported
SCSI from the array and will only send I/Os of that size or smaller and split anything larger) and within VMware
HW.
This is resolved in the following ESXi releases:
• ESXi 6.0, Patch Release ESXi600-201909001

• ESXi 6.5, Patch Release ESXi650-201811002
• ESXi 6.7 Update 1 Release
• ESXi 7.0 all releases
If you are not running one of these newer releases, it is necessary to reduce the ESXi parameter
Disk.DiskMaxIOSize from the default of 32 MB (32,768 KB) down to 4 MB (4,096 KB) or less.
The above scenarios are only applicable if the VMs reside on a Pure Storage FlashArray. If you
have VMs in your environment that are not on a Pure Storage FlashArray please consult with
your vendor to verify if any changes are required.
If this is not configured for ESXi hosts running EFI-enabled VMs, the virtual machine will fail to properly boot. If it
is not changed on hosts running VMs being replicated by vSphere Replication, replication will fail. If it is not
changed for VMs whose applications are sending requests larger than 4MB, the larger I/O requests will fail which
results in the application failing as well.
12
This should be set on every ESXi host in the cluster that VMs may have access to, in order to ensure vMotion is
successful from one ESXi host to another. If none of the above circumstances apply to your environment then
this value can remain at the default. There is no known performance impact by changing this value.
For more detail on this change, please refer to the VMware KB article here:
https://kb.vmware.com/s/article/2137402
BEST PRACTICE: Upgrade ESXi to a release that adheres to the maximum supported SCSI size
from the FlashArray.
VAAI Configuration
The VMware API for Array Integration (VAAI) primitives offer a way to offload and accelerate certain operations
in a VMware environment.
Pure Storage requires that all VAAI features be enabled on every ESXi host that is using FlashArray storage.
Disabling VAAI features can greatly reduce the efficiency and performance of FlashArray storage in ESXi
environments.
All VAAI features are enabled by default (set to 1) in ESXi 5.x and later, so no action is typically required. Though
these settings can be verified via the vSphere Web Client or CLI tools.
1. WRITE SAME—DataMover.HardwareAcceleratedInit
2. XCOPY—DataMover.HardwareAcceleratedMove
3. ATOMIC TEST & SET— VMFSHardwareAcceleratedLocking
13
BEST PRACTICE: Keep VAAI enabled. DataMover.HardwareAcceleratedInit,
DataMover.HardwareAcceleratedMove, and VMFS3.HardwareAcceleratedLocking
In order to provide a more efficient heart-beating mechanism for datastores VMware introduced a new host-
wide setting called /VMFS3/UseATSForHBOnVMFS5. In VMware’s own words:
“A change in the VMFS heartbeat update method was introduced in ESXi 5.5 Update 2, to help optimize the
VMFS heartbeat process. Whereas the legacy method involves plain SCSI reads and writes with the VMware
ESXi kernel handling validation, the new method offloads the validation step to the storage system.“
Pure Storage recommends keeping this value on whenever possible. That being said, it is a host wide setting,
and it can possibly affect storage arrays from other vendors negatively.
Read the VMware KB article here:
ESXi host loses connectivity to a VMFS3 and VMFS5 datastore
14
Pure Storage is NOT susceptible to this issue, but in the case of the presence of an affected array from another
vendor, it might be necessary to turn this off. In this case, Pure Storage supports disabling this value and
reverting to traditional heart-beating mechanisms.
BEST PRACTICE: Keep VMFS3.UseATSForHBOnVMFS5 enabled—this is preferred. If another

vendor is present and prefers it to be disabled, it is supported by Pure Storage to disable it.
For additional information please refer to VMware Storage APIs for Array Integration with the Pure Storage
FlashArray document.
iSCSI Configuration
Just like any other array that supports iSCSI, Pure Storage recommends the following changes to an iSCSI-
based vSphere environment for the best performance.
15
For a detailed walkthrough of setting up iSCSI on VMware ESXi and on the FlashArray please refer to the
following VMware white paper. This is required reading for any VMware/iSCSI user:
https://core.vmware.com/resource/best-practices-running-vmware-vsphere-iscsi
Set Login Timeout to a Larger Value

For example, to set the Login Timeout value to 30 seconds, use commands similar to the following:
1. Log in to the vSphere Web Client and select the host under Hosts and Clusters.
2. Navigate to the Manage tab.
3. Select the Storage option.
4. Under Storage Adapters, select the iSCSI vmhba to be modified.
5. Select Advanced and change the Login Timeout parameter. This can be done on the iSCSI adapter itself or
on a specific target.
The default Login Timeout value is 5 seconds and the maximum value is 60 seconds.
BEST PRACTICE: Set iSCSI Login Timeout for FlashArray targets to 30 seconds. A higher value
is supported but not necessary.
Disable DelayedAck
DelayedAck is an advanced iSCSI option that allows or disallows an iSCSI initiator to delay acknowledgment of
received data packets.
Disabling DelayedAck:
1. Log in to the vSphere Web Client and select the host under Hosts and Clusters.
2. Navigate to the Configure tab.
3. Select the Storage option.
4. Under Storage Adapters, select the iSCSI vmhba to be modified.
Navigate to Advanced Options and modify the DelayedAck setting by using the option that best matches your
requirements, as follows:
Option 1: Modify the DelayedAck setting on a particular discovery address (recommended) as follows:
1. Select Targets.
2. On a discovery address, select the Dynamic Discovery tab.
3. Select the iSCSI server.
4. Click Advanced.
16
5. Change DelayedAck to false.
Option 2: Modify the DelayedAck setting on a specific target as follows:
1. Select Targets.
2. Select the Static Discovery tab.
3. Select the iSCSI server and click Advanced.
Option 3: Modify the DelayedAck setting globally for the iSCSI adapter as follows:
1. Select the Advanced Options tab and click Advanced.

DelayedAck is highly recommended to be disabled, but is not absolutely required by Pure Storage. In highly-
congested networks, if packets are lost, or simply take too long to be acknowledged, due to that congestion,
performance can drop. If DelayedAck is enabled, where not every packet is acknowledged at once (instead one
acknowledgment is sent per so many packets) far more re-transmission can occur, further exacerbating
congestion. This can lead to continually decreasing performance until congestion clears. Since DelayedAck can
contribute to this it is recommended to disable it in order to greatly reduce the effect of congested networks
and packet retransmission.
Enabling jumbo frames can further harm this since packets that are retransmitted are far larger. If jumbo frames
are enabled, it is absolutely recommended to disable DelayedAck.
See the following VMware KB for more information:
http://kb.vmware.com/selfservice/microsites/
search.do?language=en_US&cmd=displayKC&externalId=1002598
BEST PRACTICE: Disable DelayedAck for FlashArray iSCSI targets.
iSCSI Port Binding

For software iSCSI initiators, without additional configuration the default behavior for iSCSI pathing is for ESXi to
leverage its routing tables to identify a path to its configured iSCSI targets. Without solid understanding of
network configuration and routing behaviors, this can lead to unpredictable pathing and/or path unavailability in
a hardware failure. To configure predictable and reliable path selection and failover it is necessary to configure
iSCSI port binding (iSCSI multipathing).
Configuration and detailed discussion are out of the scope of this document, but it is recommended to read
through the following VMware document that describes this and other concepts in-depth:
17
http://www.vmware.com/files/pdf/techpaper/vmware-multipathing-configuration-software-iSCSI-port-
binding.pdf
BEST PRACTICE: Use Port Binding for ESXi software iSCSI adapters when possible.
Note that ESXi 6.5 has expanded support for port binding and features such as iSCSI routing (though the use of
iSCSI routing is not usually recommended) and multiple subnets. Refer to ESXi 6.5 release notes for more
information.
Jumbo Frames
In some iSCSI environments it is required to enable jumbo frames to adhere to the network configuration
between the host and the FlashArray. Enabling jumbo frames is a cross-environment change so careful
coordination is required to ensure proper configuration. It is important to work with your networking team and
Pure Storage representatives when enabling jumbo frames. Please note that this is not a requirement for iSCSI
use on the Pure Storage FlashArray—in general, Pure Storage recommends leaving MTU at the default setting.
That being said, altering the MTU is a fully supported and is up to the discretion of the user.
1. Configure jumbo frames on the FlashArray iSCSI ports.
18
Configure jumbo frames on the physical network switch/infrastructure for each port using the relevant switch
CLI or GUI.
2. Configure jumbo frames on the physical network switch/infrastructure for each port using the relevant
switch CLI or GUI.
1. Browse to a host in the vSphere Web Client navigator.
2. Click the Configure tab and select Networking > Virtual Switches.
3. Select the switch from the vSwitch list.
4. Click the name of the VMkernel network adapter.
5. Click the pencil icon to edit.
6. Click NIC settings and set the MTU to your desired value.
7. Click OK.
8. Click the pencil icon to edit on the top to edit the vSwitch itself.
9. Set the MTU to your desired value.
10. Click OK.
Once jumbo frames are configured, verify end-to-end jumbo frame compatibility. To verify, try to ping an
address on the storage network with vmkping.
vmkping -d -s 8972 <ip address of Pure Storage iSCSI port>
If the ping operations does not return successfully, then jumbo frames is not properly configured in ESXi, the
networking devices, and/or the FlashArray port.
Challenge-Handshake Authentication Protocol (CHAP)

iSCSI CHAP is supported on the FlashArray for unidirectional or bidirectional authentication. Enabling CHAP is
optional and up to the discretion of the user. Please refer to the following post for a detailed walkthrough:
http://www.codyhosterman.com/2015/03/configuring-iscsi-chap-in-vmware-with-the-flasharray/
19
Please note that iSCSI CHAP is not currently supported with dynamic iSCSI targets on the
FlashArray. If CHAP is going to be used, you MUST configure your iSCSI FlashArray targets as
static targets.
iSCSI Failover Times

A common question encountered here at Pure Storage is why extended pauses in I/O are noted during specific
operations or tests when utilizing the iSCSI protocol. Often times the underlying reasons for these pauses in I/O
are a result of a network cable being disconnected, a misbehaving switch port, or a failover of the backend
storage array; though this list is certainly not exhaustive.
When the default configuration for iSCSI is in use with VMware ESXi the delay for these events will generally be
25-35 seconds. While the majority of environments are able to successfully recover from these events
unscathed this is not true for all environments. On a handful of occasions, there have been environments that
contain applications that need faster recovery times. Without these faster recovery times, I/O failures have been
noted and manual recovery efforts were required to bring the environment back online.
While Pure Storage's official best practice is to utilize default iSCSI configuration for failover times we also
understand that not all environments are created equal. As such we do support modifying the necessary iSCSI
advanced parameters to decrease failover times for sensitive applications.
Recovery times are controlled by the following 3 iSCSI advanced parameters:
20
Name Current Default Min Max Settable Inherit
-------------------- ---------- ---------- --- -------- -------- -------
NoopOutInterval 15 15 1 60 true false
NoopOutTimeout 10 10 10 30 true true
RecoveryTimeout 10 10 1 120 true true
To better understand how these parameters are used in iSCSI recovery efforts it is recommended you read the
following blog posts for deeper insight:
iSCSI: A 25-second pause in I/O during a single link loss? What gives?
iSCSI Advanced Settings
Once a thorough review of these iSCSI options have been investigated, additional testing within your own
environment is strongly recommended to ensure no additional issues are introduced as a result of these
changes.
Network Time Protocol (NTP)

No matter how perfect an environment is configured there will always come a time where troubleshooting an
issue will be required. This is inevitable when dealing with large and complex environments. One way to help
alleviate some of the stress that comes with troubleshooting is ensuring that the Network Time Protocol (NTP) is
enabled on all components in the environment. NTP will ensure that the timestamps for servers, arrays,
switches, etc are all aligned and in-sync. It is for this reason that Pure Storage recommends as a best practice
that NTP be enabled and configured on all components.
Please refer to VMware KB Configuring Network Time Protocol (NTP) on an ESXi host using the vSphere
Client for steps on how to configure NTP on your ESXi hosts.
Often times the VMware vCenter Server is configured to sync time with the ESXi host it resides
on. If you do not use this option please ensure the vCenter Server has NTP properly configured
and enabled as well.
Remote Syslog Server

Another helpful tool in the toolbox of troubleshooting is having a remote syslog server configured. There may be
times where an investigation is required in the environment but when attempting to review the logs it is
discovered that they are no longer available. Often times this is a result of the increased logging that happened
during the time of the issue. The increased logging leads to thresholds for file size and counts being exceeded
and thus the older logs are automatically deleted as a result.
21
Pure Storage recommends the use of the VMware vRealize Log Insight OVA. This provides for a quick and easy
integration for the ESXi hosts and vCenter. Additionally, the Pure Storage Content Pack can be used with
vRealize Log Insight which provides a single logging destination for both the vSphere and Pure Storage
environments.
Configuring vCenter Server and ESXi with Log Insight

As explained above, configuring vCenter Server and ESXi is a relatively quick and simple process.
• Login to VMware vRealize Log Insight.

• Click on Administration.
• Under Integration click on vSphere.
• Click + Add vCenter Server.
• Fill in the applicable vCenter Server information and Test Connection.
• Ensure the following boxes are checked:
◦ Collect vCenter Server events, tasks, and alarms
◦ Configure ESXi hosts to send logs to Log Insight
• Click Save to commit all of the requested changes.
The following screenshot is applicable for vRealize Log Insight 8.x. If you have an earlier
version of Log Insight then you can refer to the VMware documentation here on how to
properly configure vCenter and ESXi.
22
Additional Remote syslog Options
It is understood that not every customer or environment will have vRealize Log Insight installed or available. If
your environment takes advantage of a different solution then please refer to the third party documentation on
how the best way to integrate it with your vSphere environment. You can also refer to VMware's Knowledge
Base article Configuring syslog on ESXi for additional options and configuration information.
23

ESXi Host Configuration

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ESXi Host Configuration

Uploaded by

Copyright:

Available Formats

KNOWLEDGE - OFFLINE PDF COPY

ESXi Host Configuration

For the latest information, please visit

VMware Native Multipathing Plugin (NMP) Configuration

Round Robin Path Selection Policy

The I/O Operations Limit

This recommendation is made for a few reasons:

ESXi Express Patch 5 or 6.5 Update 1 and later

For information, refer to this blog post:

Configuring Round Robin and the I/O Operations Limit

This must be repeated for each ESXi host.

Connect-VIServer -Server <vCenter> -Credential (Get-Credential)

Furthermore, this can be configured using vSphere Host Profiles:

Configuring Round Robin and the Latency Based Sub-Policy

Connect-VIServer -Server <vCenter> -Credential (Get-Credential)

esxcli storage nmp psp roundrobin deviceconfig set --type=latency --device=naa.

This consists of a few steps:

1. Verifying proper multipathing settings in ESXi.

esxcli storage nmp device list -d naa.<device NAA>

Additionally, it is also possible to check multipathing from the FlashArray.

1. The host name.

This is resolved in the following ESXi releases:

• ESXi 6.0, Patch Release ESXi600-201909001

Read the VMware KB article here:

ESXi host loses connectivity to a VMFS3 and VMFS5 datastore

BEST PRACTICE: Keep VMFS3.UseATSForHBOnVMFS5 enabled—this is preferred. If another

Set Login Timeout to a Larger Value

Option 2: Modify the DelayedAck setting on a specific target as follows:

1. Select the Advanced Options tab and click Advanced.

See the following VMware KB for more information:

BEST PRACTICE: Disable DelayedAck for FlashArray iSCSI targets.

iSCSI Port Binding

1. Configure jumbo frames on the FlashArray iSCSI ports.

vmkping -d -s 8972 <ip address of Pure Storage iSCSI port>

Challenge-Handshake Authentication Protocol (CHAP)

iSCSI Failover Times

Recovery times are controlled by the following 3 iSCSI advanced parameters:

iSCSI Advanced Settings

Network Time Protocol (NTP)

Remote Syslog Server

Configuring vCenter Server and ESXi with Log Insight

• Login to VMware vRealize Log Insight.

You might also like