V6 Server VMware ESXi System

Best Practice

Issue 02
Date 2023-06-12

xFusion Digital Technologies Co., Ltd.

Copyright © xFusion Digital Technologies Co., Ltd. 2023. All rights reserved.
No part of this document may be reproduced or transmitted in any form or by any means without prior written
consent of xFusion Digital Technologies Co., Ltd.

Trademarks and Permissions

and other xFusion trademarks are trademarks of xFusion Digital Technologies Co., Ltd.
All other trademarks and trade names mentioned in this document are the property of their respective
holders.

Notice
In this document, "xFusion" is used to refer to "xFusion Digital Technologies Co., Ltd." for concise description
and easy understanding, which does not mean that "xFusion" may have any other meaning. Any "xFusion"
mentioned or described hereof may not be understood as any meaning other than "xFusion Digital
Technologies Co., Ltd.", and xFusion Digital Technologies Co., Ltd. shall not bear any liability resulting from
the use of "xFusion".

The purchased products, services and features are stipulated by the contract made between xFusion and
the customer. All or part of the products, services and features described in this document may not be within
the purchase scope or the usage scope. Unless otherwise specified in the contract, all statements,
information, and recommendations in this document are provided "AS IS" without warranties, guarantees or
representations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in the
preparation of this document to ensure accuracy of the contents, but all statements, information, and
recommendations in this document do not constitute a warranty of any kind, express or implied.

xFusion Digital Technologies Co., Ltd.

Address: 9th Floor, Building 1, Zensun Boya Square, Longzihu Wisdom Island
Zhengdong New District 450046
Zhengzhou, Henan Province
People's Republic of China

Website: https://www.xfusion.com

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. i

V6 Server VMware ESXi System
Best Practice About This Document

About This Document

Purpose
This document describes how to use the VMware ESXi system on servers to achieve
optimal performance and stability.

Intended Audience
This document is intended for:

● Technical support engineers

● Enterprise administrators

Symbol Conventions
The symbols that may be found in this document are defined as follows:

Symbol Description

Indicates a hazard with a high level of risk which, if not avoided,

will result in death or serious injury.

Indicates a hazard with a medium level of risk which, if not

avoided, will result in death or serious injury.

Indicates a hazard with a low level of risk which, if not avoided,

could result in minor or moderate injury.

Indicates a potentially hazardous situation which, if not avoided,

could result in equipment damage, data loss, performance
deterioration, or unanticipated results.
NOTICE is used to address practices not related to personal
injury.

Supplements the important information in the main text.

NOTE is used to address information not related to personal
injury, equipment damage, and environment deterioration.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. ii

V6 Server VMware ESXi System
Best Practice About This Document

Change History
Issue Release Date Description

02 2023-06-12 This issue is the second official release.

Updated 1.1 Sever Model.

01 2022-06-30 This issue is the first official release.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. iii

V6 Server VMware ESXi System
Best Practice Contents

Contents

About This Document.........................................................................................................ii

1 Overview........................................................................................................................... 1
1.1 Sever Model.................................................................................................................................................1
1.2 OS................................................................................................................................................................1

2 Preparing for the Deployment........................................................................................ 2

2.1 Checking Server Compatibility.....................................................................................................................2
2.2 Confirming the Server Version.....................................................................................................................2
2.2.1 Checking the Server Software Information............................................................................................... 2
2.2.2 Obtaining the Upgrade Guide................................................................................................................... 3

3 Deploying the VMware ESXi System..............................................................................4

3.1 Preparing for the Installation........................................................................................................................4
3.1.1 Loading Default BIOS Settings................................................................................................................. 4
3.1.2 Configuring a RAID Controller Card..........................................................................................................5
3.2 Installing the VMware ESXi System.............................................................................................................5

4 Performing Operations After the Deployment.............................................................. 6

4.1 Checking the Mapping Between the Firmware and Drivers.........................................................................6
4.2 Upgrading the Firmware and Drivers........................................................................................................... 7
4.3 Deploying the Fault Locating Method.......................................................................................................... 7
4.4 Optimizing the VMware ESXi System..........................................................................................................7
4.4.1 Setting Power Management Policy...........................................................................................................8
4.4.2 Disabling Power.CStateMaxLatency in the System.................................................................................. 8
4.4.3 Using NTP Server to Synchronize Host Time...........................................................................................9

5 Handling Exceptions and Collecting Information....................................................... 11

5.1 Collecting VMware ESXi System Logs...................................................................................................... 11
5.2 Collecting Server Hardware Logs.............................................................................................................. 11
5.2.1 Obtaining FusionServer Tools................................................................................................................. 11
5.2.2 Using FusionServer Tools.......................................................................................................................12

6 Known Issues.................................................................................................................13
6.1 Incompatibility Between Monitor MWAIT and EVC....................................................................................13
6.2 Exceptions Due to Lack of Necessary Drivers During System Installation................................................ 14

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. iv

V6 Server VMware ESXi System
Best Practice Contents

6.3 Upgrading VMware ESXi Patches............................................................................................................. 14

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. v

V6 Server VMware ESXi System
Best Practice 1 Overview

1 Overview

1.1 Sever Model

1.2 OS

1.1 Sever Model

This document applies to V6 rack, blade, high-density and PoD node servers.

1.2 OS
This document applies to VMware ESXi 7.0 U1 and later.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 1

V6 Server VMware ESXi System
Best Practice 2 Preparing for the Deployment

2 Preparing for the Deployment

2.1 Checking Server Compatibility

2.2 Confirming the Server Version

2.1 Checking Server Compatibility

Before deploying the OS, check the server compatibility list. Make sure that the OS
version you are using is compatible with the servers.
Use Compatibility Checker to check OSs supported by the servers.

2.2 Confirming the Server Version

Before deploying the OS, check that the components such as BIOS and BMC of the
server have been updated to the latest versions.
Access Technical Support, select a server model, and download server software
and upgrade guides.

2.2.1 Checking the Server Software Information

To view or obtain the server software, perform the following operations:
1. Visit Technical Support.
2. Select the corresponding server model.
3. Choose the Software Download tab page.
4. Click the required release software version path.
5. View or download the required software files.
The following uses the 2288H V6 as an example.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 2

V6 Server VMware ESXi System
Best Practice 2 Preparing for the Deployment

2.2.2 Obtaining the Upgrade Guide

To obtain the server upgrade guide, perform the following operations:
1. Visit Technical Support.
2. Select the corresponding server model.
3. On the Documentation tab page, choose Installation & Upgrade > Upgrade
Guide.
4. View the required upgrade guide.
The following uses the 2288H V6 as an example.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 3

V6 Server VMware ESXi System
Best Practice 3 Deploying the VMware ESXi System

3 Deploying the VMware ESXi System

3.1 Preparing for the Installation

3.2 Installing the VMware ESXi System

3.1 Preparing for the Installation

3.1.1 Loading Default BIOS Settings

The servers have been set to the optimal factory defaults before shipment. You are
advised to use the default BIOS settings. If you have modified the BIOS settings of
the servers prior to the deployment, or if you are not sure whether the BIOS settings
are the default values, you can press Delete during the BIOS power-on self test
(POST) process to launch the BIOS setup menu, and press F9 to load the default
settings. Then, press F10 to save the settings.

The following are the recommended BIOS settings for the VMware ESXi system. You
can access the BIOS setup menu to check whether your settings have adopted the
recommended values.

NO TE

The BIOS option names may vary depending on the server model. The following uses 1288H
V6 as an example. For details, see the corresponding BIOS Parameter Reference of each
server.
To obtain the BIOS Parameter Reference, perform the following operations:
1. Visit Technical Support.
2. Select the corresponding server model.
3. On the Documentation tab page, choose Reference Guides > Parameter Reference.
4. View the required BIOS Parameter Reference.

BIOS Option Recommended Value

Hyper-Threading Enabled

Enable Monitor MWAIT[1] Disabled

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 4

V6 Server VMware ESXi System
Best Practice 3 Deploying the VMware ESXi System

BIOS Option Recommended Value

VMX Enabled

PCIe SR-IOV Enabled

Intel(R) VT for Directed I/O (VT-d) Enabled

Interrupt Remapping Enabled

PCI 64-Bit Resource Allocation Enabled

Hardware Prefetcher Enabled

Adjacent Cache Prefetch Enabled

DCU Streamer Prefetcher Enabled

DCU IP Prefetcher Enabled

Note 1: For details about the impact of the value of Enable Monitor MWAIT on
EVC compatibility, see section 6.1 Incompatibility Between Monitor MWAIT and
EVC.

3.1.2 Configuring a RAID Controller Card

Configure RAID arrays if required before installing the OS.
● Configure RAID arrays.
The method for configuring RAID arrays varies with the RAID controller card
type. For details, see V6 Server RAID Controller Card User Guide.
● Set the boot devices.
If multiple RAID arrays or a RAID array with multiple virtual drives (VDs) is
configured, you must set the boot device. Otherwise, the OS installation my
become abnormal. The method for setting the boot device varies with the RAID
controller card type. For details, see V6 Server RAID Controller Card User
Guide.

3.2 Installing the VMware ESXi System

You are advised to install the VMware ESXi system by using Smart Provisioning . For
details, see "Installing an OS Using Smart Provisioning" in the Server OS
Installation Guide. Alternatively, you can install the VMware ESXi system by using
the VMware ISO.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 5

V6 Server VMware ESXi System
Best Practice 4 Performing Operations After the Deployment

4 Performing Operations After the

Deployment

4.1 Checking the Mapping Between the Firmware and Drivers

4.2 Upgrading the Firmware and Drivers
4.3 Deploying the Fault Locating Method
4.4 Optimizing the VMware ESXi System

4.1 Checking the Mapping Between the Firmware and

Drivers
The drivers of certain cards on servers require firmware of corresponding versions.
After installing the VMware ESXi system, check the mapping between the firmware
and drivers by using the following method.

Step 1 Run the install.sh script in the iDriver package by using the following command:
sh install.sh

Step 2 Choose 2: To check the current module information and driver version.

Then, an interface similar to the following figure will be displayed, showing the
firmware and driver mapping check results.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 6

V6 Server VMware ESXi System
Best Practice 4 Performing Operations After the Deployment

Among the command output:

● CurrentDriverVer indicates the driver version currently used in the system.
● NewDriverVersion indicates the driver version recommended for the system.
● CurrentFirmwareVer indicates the current firmware version.
● NewFirmwareVersion indicates the firmware version recommended for the
system.
If you find that CurrentFirmwareVer is different from NewFirmwareVersion, or that
CurrentDriverVer is different from NewDriverVersion, you need to upgrade the
firmware and/or the drivers.

----End

4.2 Upgrading the Firmware and Drivers

In section 4.1 Checking the Mapping Between the Firmware and Drivers, if you
find that the firmware or driver version is not the latest, upgrade the firmware and/or
the drivers by referring to "Installing and Upgrading Drivers and Firmware" in the
Server OS Installation Guide.

4.3 Deploying the Fault Locating Method

Collecting as many logs as possible when servers become faulty is helpful for quickly
analyzing and locating the fault. You are advised to configure Serial Over LAN (SOL)
for the servers, and adjust the size of the ESXi dump partition.
For details about the configuration methods, see "Locating OS Faults" in the Server
OS Installation Guide.

4.4 Optimizing the VMware ESXi System

To ensure that the servers achieve optimal performance and stability with the
VMware ESXi system, you are advised to optimize the VMware ESXi system as
follows.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 7

V6 Server VMware ESXi System
Best Practice 4 Performing Operations After the Deployment

4.4.1 Setting Power Management Policy

Choose Manage > Hardware > Power Management > Change policy in the
VMware ESXi system. In the Change power policy window, select High
performance for Power Management Policy.

NO TE

Before setting Power Management Policy, set Hardware P-State to Native Mode on the
BIOS.

4.4.2 Disabling Power.CStateMaxLatency in the System

The servers deliver higher performance when the Power.CStateMaxLatency option
of the CPU is disabled in the VMware ESXi system. Use the following method to
disable this option.

Step 1 Choose Manage > System > Advanced settings.

Step 2 Select Power.CStateMaxLatency.

Step 3 Set Power.CStateMaxLatency to 0.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 8

V6 Server VMware ESXi System
Best Practice 4 Performing Operations After the Deployment

----End

4.4.3 Using NTP Server to Synchronize Host Time

To address time synchronization issues between ESXi hosts and other components
in the vSphere network, you can synchronize the host time and date to an NTP
server.
NTP and PTP services cannot run at the same time. You need to stop the PTP
service before starting the NTP service. In addition, when the NTP service is started,
you cannot manually configure time.

NO TE

You can set the NTP service startup policy to control the NTP service. You can also manually
change the NTP service status.

Step 1 In the VMware ESXi system, choose Manage > System > Time & date.

Step 2 Click Edit settings.

Step 3 In the Edit time configuration dialog box that is displayed, select Use Network
Time Protocol (enable NTP client) and set parameters as follows:
● NTP service startup policy: Set the value to Start and stop with host.
● NTP servers: Enter the IP address or host name of the NTP server to be used.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 9

V6 Server VMware ESXi System
Best Practice 4 Performing Operations After the Deployment

Step 4 Click Save.

----End

NO TICE

Select an appropriate NTP server resource. If there are situations where

organizations or individuals need to rely on the correct time, or where an error in the
time could result in a loss or damage, contact equipment and service providers to
find out how to obtain reliable time services.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 10

V6 Server VMware ESXi System
Best Practice 5 Handling Exceptions and Collecting Information

5 Handling Exceptions and Collecting

Information

Collect the VMware ESXi system logs and server hardware logs in the case of any
exceptions when you are using the VMware ESXi system on servers.
5.1 Collecting VMware ESXi System Logs
5.2 Collecting Server Hardware Logs

5.1 Collecting VMware ESXi System Logs

● You can collect VMware ESXi system logs by using the vSphere Client or
vSphere Web Client. For details on the collection methods, visit the VMware
official knowledge base (KB):
Collecting diagnostic information for VMware ESX/ESXi (653)
● You can also collect VMware ESXi system logs by using vm-support command.
For details on the collection methods, visit the VMware official KB:
"vm-support" command in ESX/ESXi to collect diagnostic information
(1010705)

5.2 Collecting Server Hardware Logs

Use FusionServer Tools to collect server hardware logs.

5.2.1 Obtaining FusionServer Tools

Visit FusionServer Tools, click the Software Download tab, select the latest
release software version, and download the FusionServer Tools.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 11

V6 Server VMware ESXi System
Best Practice 5 Handling Exceptions and Collecting Information

Figure 5-1 Obtaining the FusionServer Tools 1

Figure 5-2 Obtaining the FusionServer Tools 2

5.2.2 Using FusionServer Tools

1. Install FusionServer Tools. For details, see section "Installing FusionServer
Tools" in FusionServer Tools User Guide.
2. Import the log collection tool (FusionServer Tools_X.X.X_ServerMain.zip).
3. Collect server logs. For details, see section "Collecting Server Logs" in
FusionServer Tools User Guide.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 12

V6 Server VMware ESXi System
Best Practice 6 Known Issues

6 Known Issues

6.1 Incompatibility Between Monitor MWAIT and EVC

6.2 Exceptions Due to Lack of Necessary Drivers During System Installation
6.3 Upgrading VMware ESXi Patches

6.1 Incompatibility Between Monitor MWAIT and EVC

Symptom
In the BIOS of V6 servers, the default value of the Monitor MWAIT is Disabled.

In a cluster where the Enhanced Vmotion Compatibility (EVC) mode is enabled, the
VMware ESXi host may fail to be added to the cluster.

Solution
If this problem occurs, you are advised to check whether the servers in the cluster all
use CPUs of architectures of the same generation. If the server CPU architectures
are of the same generation, the EVC feature does not take effect. In such a case, you
are advised to disable the EVC feature of the cluster. Disabling the EVC feature does
not affect practical usage.

If servers in the cluster are of different CPU architecture generations and the EVC
feature needs to be enabled, set Monitor MWAIT to Enabled on the BIOS.

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 13

V6 Server VMware ESXi System
Best Practice 6 Known Issues

NO TE

For details about the EVC feature of the VMware ESXi system, visit the VMware official KB:
Enhanced vMotion Compatibility (EVC) processor support (1003212)

6.2 Exceptions Due to Lack of Necessary Drivers During

System Installation
During system installation, exceptions may occur due to lack of necessary drivers in
the system. In such a case, refer to "Installing an OS by Creating an Installation
Source" in the Server OS Installation Guide to customize the VMware ESXi system
ISO to install the system.

6.3 Upgrading VMware ESXi Patches

If you have installed VMware ESXi drivers as described in section 4.2 Upgrading the
Firmware and Drivers, exercise caution when upgrading VMware ESXi patches. Do
not upgrade patches by running the install command, for example, esxcli software
vib install -d "/vmfs/volumes/Datastore/DirectoryName/PatchName.zip".
This method may overwrite existing drivers.
You are advised to run the update command, for example, esxcli software vib
update -d "/vmfs/volumes/Datastore/DirectoryName/PatchName.zip".
For details about upgrading VMware ESXi patches, see "esxcli software vib"
commands to patch an ESXi 5.x/6.x host (2008939).

Issue 02 (2023-06-12) Copyright © xFusion Digital Technologies Co., Ltd. 14