vSphere 4.1: Delta to 4.

0
Tech Sharing for Partners

Iwan ‘e1’ Rahabok, Senior Systems Consultant

e1@vmware.com | virtual-red-dot.blogspot.com | tinyurl.com/SGP-User-Group | facebook.com/e1ang
August 2010

Confidential
© 2009 VMware Inc. All rights reserved
 Audience Assumption

This is a level 200 - 300 presentation.

It assumes:
• Good understanding of vCenter 4, ESX 4, ESXi 4.
 Preferably hands-on
 We will only cover the delta between 4.1 and 4.0
• Overview understanding of related products like VUM, Data Recovery, SRM,
View, Nexus, Chargeback, CapacityIQ, vShieldZones, etc
• Good understanding of related storage, server, network technology
Target audience
• VMware Specialist: SE + Delivery from partners

2 Confidential
 Agenda

New features
• Server
• Storage
• Network
• Management
Upgrade

3 Confidential
 4.1 New Feature (over 4.0, not 3.5): Server
Features Design Cost Scalability Performance Availability Security Manageability
ESXi: scripted install   
ESXi: SAN Boot   
Memory compression    
Serial Port Concentrator   
USB Device  
MS Cluster support 
HA Health Check  
HA: more VM per cluster  
FT enhancements  
DRS/HA/FT integration   
FT: enhanced logging 

4 Confidential
 4.1 New Feature (over 4.0, not 3.5): Server
Features Design Cost Scalability Performance Availability Security Manageability
vMotion enhancements   
Power Management &
Charts 
More VM per host?

Reduced RAM overhead 

Host Affinity Rules  
AD integration   
Multi-core VM 
Local/Remote Console 
Total Lockdown Mode 
VMware Tools scripting

5 Confidential
 4.1 New Feature (over 4.0, not 3.5): Storage

Features Design Cost Scalability Performance Availability Security Manageability

API for Array Integration    

vscsiStats in ESXi 
Storage I/O Control   
iSCSI Hardware Offload  
VMware Data Recovery   
VADP enhancements 
Boot from iSCSI Software 
Pluggable Storage Arch

VMFS enhancements 
Storage statistics 
Paravirtualised SCSI

Improved performance

8 GB FC support 

6 Confidential
 4.1 New Feature (over 4.0, not 3.5): Network

Features Design Cost Scalability Performance Availability Security Manageability

Network I/O Control   

IPv6 Enhancements  
Load-based Teaming 
vNIC enhancements 
Nexus 1000V v2.0 
Distributed Switch 

7 Confidential
 4.1 New Feature: Management

Component New Features

vMA AD authentication
Host Profiles Cisco, AD, Tech Support Mode
vCLI & PowerShell A set of new vCLI commands
vCO 64 bit. Improved performance.
3rd party patching, provisioning, upgrading. Push update on critical
VMware Update Manager
notifications
Licence Reporting Manager
Faster performance, 64 bit, more VM per host, more hosts per vCenter,
vCenter
bigger vCenter,
vCenter LinkedMode 3x more VM
Per-VM pricing. IP customization for Windows 7 and Win08 R2. Faster
Site Recovery Manager 4.1
recovery time for iSCSI . 64-bit only. vDS support.
Error Reporting Submit error to VMware.com
Partner plug-in Updated vCenter plug-ins from partners (Server, Storage, etc)
Converter Convert to thin while converting. Hyper-V import
Performance Charts New charts, new counters, especially Storage related

8 Confidential
 Builds:
• ESX build 260247
• VC build 258902
Some stats:
• 4000 development weeks were spent to get to FC
• 5100 QA weeks were spent to get to FC
• 872 beta customers downloaded and tried it out
• 2012 servers, 2277 storage arrays, and 2170 IO devices are already on the
HCL
 

9 Confidential
 Consulting Services: Kit

The vSphere Fundamentals services kit

• Includes core services enablement materials for
vSphere Jumpstarts, Upgrades, Converter/P2V
and PoCs. 
• The update reflects what’s new in vSphere 4.1 -
including new resource limits, memory
compression, Storage IO Control, vNetwork
Traffic Management, and vSphere Active For delivery partner:
Directory Integration.  Please
• The kit is intended for use by PSO Consultants, download this.
TAMs, and SEs to help with delivering services
engagements, PoCs, or knowledge transfer
sessions with customers. 
• Located at Partner Central – Services IP Assets
 https://na6.salesforce.com/sfc/#version?selectedD
ocumentId=069800000000SSi

10 Confidential
 4.1 New Features: Server

Confidential
© 2009 VMware Inc. All rights reserved
 PXE Boot Retry

Virtual Machine -> Edit Settings -> Options -> Boot Options
• Failed Boot Recovery disabled by default
• Enable and set the automatically retry boot after X Seconds

12 Confidential
12
 Wide NUMA Support

Wide VM
• Wide-VM is defined as a VM that has more vCPUs than the available cores on
a NUMA node.
• A 5-vCPU VM in a quad-core server
• Only the cores count, and hyperthreading threads don’t
ESX 4.1 scheduler introduces wide-VM NUMA support
• Improves memory locality for memory-intensive workloads.
• Based on testing with micro benchmarks, the performance benefit can be up
to 11–17%.
How it works
• ESX 4.1 allows wide-VMs to take advantage of NUMA management. NUMA
management means that a VM is assigned a home node where memory is
allocated and vCPUs are scheduled. By scheduling vCPUs on a NUMA node
where memory is allocated, the memory accesses become local, which is
faster than remote accesses

13 Confidential
 ESXi
Enhancements to ESXi. Not applicable to ESX

14 Confidential
 Transitioning to ESXi

ESXi is our architecture

going forward

15 Confidential
 Moving toward ESXi
Permalink to: VMware ESX and ESXi 4.1 Comparison

Service Console (COS)

Management Agents Agentless vAPI-based

Hardware Agents Agentless CIM-based

vCLI, PowerCLI
Commands for
configuration and
diagnostics
Local Support Console

CIM API vSphere API

Infrastructure
Service Agents Native Agents:
NTP, Syslog, SNMP
“Classic” VMware ESX VMware ESXi

16 Confidential
 Software Inventory - Connected to ESXi/ESX

From vSphere 4.1

Before

Enumerate instance of
CIM_SoftwareIdentity

Enhanced CIM provider now displays great detail on installed software bundles.

17 Confidential
 Software Inventory – Connected to vCenter
Before From vSphere 4.1

Enumerate instance of
CIM_SoftwareIdentity

• Enhanced CIM provider now displays great detail on installed software bundles.

18 Confidential
18
 Additional Deployment Option

Boot From SAN

• Fully supported in ESXi 4.1
• Was only experimentally supported in ESXi 4.0
• Boot from SAN supported for FC, iSCSI, and FCoE
• ESX and ESXi have different requirement:

iBFT (Boot Firmware Table) required

• The host must have an iSCSI boot capable NIC that supports the iSCSI iBFT
format.
• iBFT is a method of communicating parameters about the iSCSI boot device
to an OS

19 Confidential
 Additional Deployment Option

Scripted Installation
• Numerous choices for installation
 Installer booted from
­ CD-ROM (default)
­ Preboot Execution Environment (PXE)
 ESXi Installation image on
­ CD-ROM (default), HTTP/S, FTP, NFS
 Script can be stored and accessed
­ Within the ESXi Installer ramdisk
­ On the installation CD-ROM
­ HTTP / HTTPS, FTP, NFS
 Config script (“ks.cfg”) can include
­ Preinstall
­ Postinstall
­ First boot
• Cannot use scripted installation to install to a USB device
20 Confidential
 PXE Boot

Requirements
• PXE-capable NIC.
• DHCP Server (IPv4). Use existing one.
• Media depot + TFTP server + gPXE
 A server hosting the entire content of ESXi
media.
 Protocal: HTTP/HTTPS, FTP, or NFS server.
 OS: Windows/Linux server.
Info
• We recommend the method that uses gPXE.
If not, you might experience issues while
booting the ESXi installer on a heavily loaded
Network.
• TFTP is a light-weight version of the FTP
service, and is typically used only for network
booting systems or loading firmware on
network devices such as routers.
21 Confidential
 PXE boot

PXE uses DHCP and Trivial File Transfer Protocol (TFTP) to bootstrap an OS
over network.
How it works
• A host makes a DHCP request to configure its NIC.
• A host downloads and executes a kernel and support files. PXE booting the
installer provides only the first step to installing ESXi.
• To complete the installation, you must provide the contents of the ESXi DVD
• Once ESXi installer is booted, it works like a DVD-based installation, except
that the location of the ESXi installation media must be specified.

22 Confidential
 Additional Deployment Option

23 Confidential
 Sample ks.cfg file

# Accept the EULA (End User Licence Agreement)

vmaccepteula
# Set the root password to vmware123
rootpw vmware123
# Install the ESXi image from CDROM
install cdrom
# Auto partition the first disk – if a VMFS exists it will overwrite it.
autopart --firstdisk --overwritevmfs
# Create a partition called Foobar
# Partition the disk identified with vmhba1:c0:t1:l0 to grow to a maxsize of 4000
partition Foobar --ondisk=mpx.vmhba1:C0:T1:L0 --grow –maxsize=4000
# Set up the management network on the vmnic0 using DHCP
network –bootproto=dhcp --device=vmnic0 --addvmportgroup=0
%firstboot --level=90.1 --unsupported --interpreter=busybox
# On this first boot, save the current date to a temporary file
date > /tmp/foo
# Mount an nfs share and put it at /vmfs/volumes/www
esxcfg-nas -add -host 10.20.118.5 -share /var/www www

24 Confidential
 Full Support of Tech Support Mode

There you go 
2 types
• Remote: SSH
• Local: Direct Console

25 Confidential
 Full Support of Tech Support Mode

Enter to toggle. That’s it!

• Disable/Enable
Timeout automatically
disables TSM (local and
remote)
Running sessions are not
terminated.
All commands issued in Tech
Support Mode are sent to
syslog

26 Confidential
 Full Support of Tech Support Mode

Recommended uses
• Support, troubleshooting, and break-fix
• Scripted deployment preinstall, postinstall, and first boot scripts
Discouraged uses
• Any other scripts
• Running commands/scripts periodically (cron jobs)
• Leaving open for routine access or permanent SSH connection

Admin will be
notified when active

27 Confidential
 Full Support of Tech Support Mode

We can also enable it via GUI

Can enable in vCenter or DCUI

Enable/Disable

28 Confidential
 Security Banner

A message that is displayed on the direct console Welcome screen.

29 Confidential
 Total Lockdown

30 Confidential
 Total Lockdown

Ability to totally control local access via vCenter

• DCUI
• Lockdown Mode (disallows all access except root on DCUI)
• Tech Support Mode (local and remote)
• If all configured, then no local activity possible (except pull the plugs)

31 Confidential
 Additional commands in Tech Support Mode

vscsciStats is now available in the console.

Output is raw data for histogram.
• Use spreadsheet to plot the histogram
Some use cases:
• Identify whether IO are
sequential or random
• Optimizing for IO Sizes
• Checking for disk mis-alignment
• Looking at storage latency in more
details

32 Confidential
 Additional commands in Tech Support Mode

Additional commands for troubleshooting

• nc (netcat)
 http://en.wikipedia.org/wiki/Netcat
• tcpdump-uw
 http://en.wikipedia.org/wiki/Tcpdump

33 Confidential
 More ESXi Services listed

More services are now shown in GUI.

• Ease of control
For example, if SSH is not running, you can turn it on from GUI.

ESXi 4.0

ESXi 4.1

34 Confidential
 ESXi Diagnostics and Troubleshooting

• During normal operations: • If things go wrong:

DCUI: misconfigs / restart m
vCenter vCLI

vSphere APIs

TSM: Advanced troubleshoo

Remote Local
Access ESXi Access

35 Confidential
 Common Enhancements for both ESX and ESXi

64 bit User World

• Running VMs with very large memory footprints implies that we need a large
address space for the VMX.
• 32-bit user worlds (VMX32) do not have sufficient address space for VMs
with large memory. 64-bit User worlds overcome this limitation.
NFS
• The number of NFS volumes supported is increased from 8 to 64.
Fiber Channel
• End-To-End Support for 8 GB (HBA, Switch & Array).
VMFS
• Version changed to 3.46. No customer visible changes. Changes related to
algorithms in the vmfs3 driver to handle new VMware APIs for Array
Integration (VAAI).

36 Confidential
 Common Enhancements for both ESX and ESXi

VMkernel TCP/IP Stack Upgrade

• Upgraded to version based on BSD 7.1.
• Result: improving FT logging, VMotion and NFS client performance.
Pluggable Storage Architecture (PSA)
• New naming convention.
• New filter plugins to support VAAI (vStorage APIs for Array Integration).
• New PSPs (Path Selection Policies) for ALUA arrays.
• New PSP from DELL for the EqualLogic arrays.

37 Confidential
 USB pass-through
New Features for both ESX/ESXi

38 Confidential
 USB Devices

2 steps:
• Add USB Controller
• Add USB Devices

39 Confidential
 USB Devices

Only devices listed on the manual is supported.

Mostly for ISV licence dongle.
A few external USB drives.
Limited list of device for now

40 Confidential
 Example 1

After vMotion, the VM will be on another

(remote) ESXi.
Communication inter-ESXi will use Mgmt
Network (ESXi has no SC network)

You cannot multi-select devices at this stage

– add them one by one.

Source:
http://vstorage.wordpress.com/2010/07/15/usb-

passthrough-in-vsphere-4-1/

41 Confidential
 Example 1

From the source

• “I have tested numerous
brands of USB mass storage
devices (Kingston, Sandisk,
Lexar, Imation) as well a
couple of of security dongles
and they all work well.”

42 Confidential
 Example 2: adding UPS

Source:

http://vninja.net/virtualization/

using-usb-pass-through-in-vsphere-4-1/

43 Confidential
 Example 2

Source:
http://vninja.net/virtualization/

using-usb-pass-through-in-vsphere-4-1/

44 Confidential
 USB Devices: Supported Devices
Device Model Device Display Name
Rainbow SafeNet Sentinel
SafeNet Sentinel Software Protection Dongle (purple)

Rainbow USB UltraPro

SafeNet Sentinel Software Protection SuperPro Dongle (gray)

Future Devices HID UNIKEY

SecuTech Unikey Software Protection Dongle

MAI KEYLOK II Software Protection Dongle Microcomputer Applications USB Device

MAI KEYLOK Fortress Software Protection Dongle (Designed for
Windows)
Philips KEYLOK Device
Note: it is not designed for Linux systems. If you connect it to
Linux systems, the connection resets frequently and can cause
unexpected behavior.
Aladdin HASP HL Drive Aladdin Knowledge HASP HL 3.21, Kingston drive
Aladdin HASP HL Basic Software Protection Dongle Aladdin Knowledge HASP HL 3.21
Aladdin HASP HL Pro Software Protection Dongle Aladdin Knowledge HASP HL 3.21
Aladdin HASP HL Max Software Protection Dongle Aladdin Knowledge HASP HL 3.21
Aladdin HASP HL Net Software Protection Dongle Aladdin Knowledge HASP HL 3.21
Aladdin HASP HL NetTime Software Protection Dongle Aladdin Knowledge HASP HL 3.21
Kingston DataTraveler 101 II 4GB Toshiba DT 101 II
Lexar JD FireFly 2GB Lexar Media JD FireFly
Western Digital My Passport Essential 250GB 2.5 HDD Western Digital External
Cables To Go USB 2.0 7-Port Hub Model# 29560 Not applicable

45 Confidential
 USB Devices

Up to 20 devices per VM. Up to 20 devices per ESX host.

1 device can only be owned by 1 VM at a given time. No sharing.
Supported
• vMotion
 Communication via the management network
• DRS
Unsupported
• DPM. DPM is not aware of the device and may turn it off. This may cause loss
of data. So disable DRS for this VM so it stays in this host only.
• Fault Tolerance
Design consideration
• Take note of situation when the ESX host is not available (planned or
unplanned downtime)

46 Confidential
 MS AD integration
New Features for both ESX/ESXi

47 Confidential
 AD Service

Provides authentication for all local services

• vSphere Client
• Other access based on vSphere API
• DCUI
• Tech Support Mode (local and remote)

Has nominal AD groups functionality

• Members of “ESX Admins” AD group have Administrative privilege
• Administrative privilege includes:
 Full Administrative role in vSphere Client and vSphere API clients
 DCUI access
 Tech Support Mode access (local and remote)

48 Confidential
 The Likewise Agent

ESX uses an agent from Likewise to connect to MS AD and to authenticate

users with their domain credentials.
The agent integrates with the VMkernel to implement the mapping for
applications such as the logon process (/bin/login) which uses a pluggable
authentication module (PAM).
As such, the agent acts as an LDAP client for authorization (join domain) and
as a Kerberos client for authentication (verify users).
• The vMA appliance also uses an agent from Likewise.
• ESX and vMA use different versions of the Likewise agent to connect to the
Domain Controller. ESX uses version 5.3 whereas vMA uses version 5.1.

49

49 Confidential
 Joining AD: Step 1

50 Confidential
 Joining AD: Step 2

1. Select “AD”

2. Click “Join Domain”

3. Join the domain. Full name.

@123.com

51 Confidential
 AD Service

A third method for joining ESX/ESXi hosts and enabling Authentication

Services to utilize AD is to configure it through Host Profiles

52 Confidential
 AD Likewise Daemons on ESX

• lwiod is the Likewise I/O Manager service - I/O services for communication.
Launched from /etc/init.d/lwiod script.
• netlogond is the Likewise Site Affinity service - detects optimal AD domain
controller, global catalogue and data caches. Launched from
/etc/init.d/netlogond script.
• lsassd is the Likewise Identity & Authentication service. It does authentication,
caching and idmap lookups. This daemon depends on the other two daemons
running. Launched from /etc/init.d/lsassd script.

root 18015 1 0 Dec08 ? 00:00:00 /sbin/lsassd --start-as-daemon

root 31944 1 0 Dec08 ? 00:00:00 /sbin/lwiod --start-as-daemon
root 31982 1 0 Dec08 ? 00:00:02 /sbin/netlogond --start-as-daemon

53 Confidential
 ESX Firewall Requirements for AD
• Certain ports in SC are automatically opened in the Firewall Configuration to
facilitate AD.
• Not applicable to ESXi

Before

After

54 Confidential
 Time Sync Requirement for AD

Time must be in sync between the ESX/ESXi server and the AD server.
For the Likewise agent to communicate over Kerberos with the domain
controller, the clock of the client must be within the domain controller's
maximum clock skew, which is 300 seconds, or 5 minutes, by default.
The recommendation would be that they share the same NTP server.

55 Confidential
 vSphere Client

Now when assigning permissions to users/groups, the list of users and groups
managed by AD can be browsed by selecting the Domain.

56 Confidential
 Info in AD

• The host should also be visible on the Domain Controller in the AD

Computers objects listing.
• Looking at the ESX Computer Properties shows a Name of RHEL (as it the
Service Console on the ESX) & Service pack of ‘Likewise Identity 5.3.0’

57 Confidential
 Memory Compression
New Features for both ESX/ESXi

58 Confidential
 Memory Compression

VMKernel implement a per-VM compression cache to store compressed guest pages.

• When a guest page (4 KB page) needs to swapped, VMKernel will first try to compress
the page. If the page can be compressed to 2 KB or less, the page will be stored in the
per-VM compression cache.
• Otherwise, the page will be swapped out to disk. If a compressed page is again
accessed by the guest, the page will decompressed online.

59 Confidential
 Changing the value of cache size

60 Confidential
 Virtual Machine Memory Compression
• Virtual Machine -> Resource Allocation
• Per-VM statistic showing compressed memory

61 Confidential
 Monitoring Compression

3 new counters introduced to monitor

• Host level, not VM level.

62 Confidential
 Power Management

63 Confidential
 Power consumption chart

Per ESX, not per cluster

Need hardware integration.
• Difference HW makes have different info

64 Confidential
 Performance Graphs – Power Consumption

• We can now track the Power consumption of VMs in real-time

 Enabled through Software Settings ->Advanced Settings -> Power -> Power.ChargeVMs

65
65 Confidential
 Host power consumption
• In some situation, may need to edit /usr/share/sensors/vmware to get
support for the host
• Different HW makers have different API.
VM power consumption
• Experimental. Off by default

66 Confidential
 ESX
Features only for ESX (not ESXi)

67 Confidential
 ESX: Service Console firewall

Changes in ESX 4.1

• ESX 4.1 introduces these additional configuration files located in
/etc/vmware/firewall/chains:
 usercustom.xml
 userdefault.xml
Relationship between the 2 files
• “user” overwrites.
• The default files custom.xml and default.xml are overridden
by usercustom.xml and userdefault.xml.
• All configuration is saved in usercustom.xml and userdefault.xml.
• Copy the original custom.xml and default.xml files.
• Use them as a template for usercustom.xml and userdefault.xml.

68 Confidential
 Cluster
HA, FT, DRS & DPM

69 Confidential
 Availability Feature Summary

HA and DRS Cluster Limitations

High Availability (HA) Diagnostic and Reliability Improvements
FT Enhancements
vMotion Enhancements
• Performance
• Usability
• Enhanced Feature Compatibility
VM-host Affinity (DRS)
DPM Enhancements
Data Recovery Enhancements

70 Confidential
 DRS: more HA-awareness

vSphere 4.1 adds logic to prevent imbalance that may not be good from HA
point of view.
Example
• 20 small VM and 2 very large VM.
• 2 ESXi hosts. Same workload with the above 20 collectively.
• vSphere 4.0 may put 20 small VM on Host A and 2 very large VM on Host B.
• From HA point of view, this may result in risks when Host A fails.
• vSphere 4.1 will try to balance the number of VM.

71 Confidential
 HA and DRS Cluster Improvements

Increased cluster limitations

• Cluster limits are now unified for HA and DRS clusters
• Increased limits for VMs/host and VMs/cluster
• Cluster limits for HA and DRS:
• 32 hosts/cluster
• 320 VMs/host (regardless of # of hosts/cluster)
• 3000 VMs/cluster
• Note that these limits also apply to post-failover scenarios. Be sure that
these limits will not be violated even after the maximum configured
number of host failovers.

72 Confidential
 HA and DRS Cluster Limit

5-host cluster, tolerate 1 host failure

• vSphere 4.1 supports 320 VMs/host
• Supports 320x5 VMs/cluster? NO
• Cluster can only support 320x4 VMs
X

5-host cluster, tolerate 2 host failures

• Supports 320x5 VMs/cluster? NO
• Cluster can only support 320x3 VMs

X X
73 Confidential
 HA Diagnostic and Reliability Improvements

HA Healthcheck Status
• HA provides an ongoing healthcheck facility to ensure that the required cluster
configuration is met at all times. Deviations result in an event or alarm on the
cluster.

Improved HA-DRS interoperability during HA failover

• DRS will perform vMotion to free up contiguous resources (i.e. on one host) so
that HA can place a VM that needs to be restarted

74 Confidential
 HA Diagnostic and Reliability Improvements

HA Operational Status
• Displays more information about the current HA operational
status, including the specific status and errors for each host in the
HA cluster.
• It shows if the host is Primary or Secondary!

75 Confidential
 HA Operational Status

Just another example 

76 Confidential
 HA: Application Awareness

Application Monitoring can restart a VM if

the heartbeats for an application it is
running are not received
Expose APIs for 3rd party app developers
Application Monitoring works much the
same way that VM Monitoring:
• If the heartbeats for an application are
not received for a specified time via
VMware Tools,
its VM is restarted.

ESXi 4.0

ESXi 4.1

77 Confidential
 Fault Tolerance

78 Confidential
 FT Enhancements

DRS
FT fully integrated with DRS
• DRS load balances FT Primary and
Secondary VMs. EVC required.
FT Primary FT Secondary
VM VM Versioning control lifts
requirement on ESX build
consistency
• Primary VM can run on host with a
different build # as Secondary VM.

Resource Pool Events for Primary VM vs.

Secondary VM differentiated
• Events logged/stored differently.

79 Confidential
 No data-loss Guarantee

vLockStep: 1 CPU step behind

Primary/backup approach
• A common approach to implementing fault-tolerant servers is the
primary/backup approach. The execution of a primary server is replicated by
a backup server. Given that the primary and backup servers execute
identically, the backup server can take over serving client requests without
any interruption or loss of state if the primary server fails

80 Confidential
 New versioning feature

FT now has a version number to determine compatibility

 Restriction to have identical ESX build # has been lifted
 Now FT checks it’s own version number to determine compatibility
 Future versions might be compatible with older ones, but possibly not vice-versa

Additional information on vSphere Client

 FT version displayed in host summary tab
 # of FT enabled VMs displayed there
 For hosts prior to ESX/ESXi 4.1, this tab
lists the host build number instead.

FT versions included in vm-support output

 /etc/vmware/ft-vmk-version:
product-version = 4.1.0
build = 235786
ft-version = 2.0.0

81 Confidential
 FT logging improvements
• FT traffic was bottlenecked to 2 Gbit/s even on 10 Gbit/s pNICs
• Improved by implementing ZeroCopy feature for FT traffic Tx, too
 For sending only (Tx)
 Instead of copying from FT buffer into pNIC/socket buffer just a link to the memory
holding the data is transferred
 Driver accesses data directly- no copy needed

82 Confidential
 FT: unsupported vSphere features

Snapshots.
• Snapshots must be removed or committed before FT can be enabled on a VM. It is not
possible to take snapshots of VMs on which FT is enabled.
Storage vMotion.
• Cannot invoke Storage vMotion for FT VM. To migrate the storage, temporarily turn
off FT, do Storage vMotion, then turn on FT.
Linked clones.
• Cannot enable FT on a VM that is a linked clone, nor can you create a linked clone
from an FT-enabled VM.
Back up.
• Cannot back up an FT VM using VCB, vStorage API for Data Protection, VMware Data
Recovery or similar backup products that require the use of a VM snapshot, as
performed by ESXi. To back up VM in this manner, first disable FT, then re-enable FT
after backup is done.
• Storage array-based snapshots do not affect FT.
Thin Provisioning, NPIV, IPv6, etc
83 Confidential
 FT: performance sample

MS Exchange 2007
• 1 core handles 2000 Heavy Online user profile
• VM CPU utilisation is only 45%. ESX is only 8%
Based on previous “generation”
• Xeon 5500, not
5600
• vSphere 4.0, not
4.1
Opportunity
• Higher uptime for
customer email
system

84 Confidential
 Integration with HA

Improved FT host management

• Move host out of vCenter
• DRS able to vMotion
FT VMs
• Warning if HA gets
disabled and
following operations
will be disabled
 Turn on FT
 Enable FT
 Power on a FT VM
 Test failover
 Test secondary restart

85 Confidential
 VM-to-Host Affinity

86 Confidential
 Background

Different servers in a datacenter is a common scenario

• Differences by memory size, CPU generation or # or type of pNICs
• Best practice up to now
 Separate different hosts in different clusters
• Workarounds
 Creating affinity/ anti-affinity rules
 Pinning a VM to a single host by disabling DRS on the VM.
• Disadvantage
 Too expensive as each cluster needed to have HA failover capacity
New feature: DRS Groups
• Host and VM groups
• Organize ESX hosts and VMs into groups
 Similar memory
 Similar usage profile
…

87 Confidential
VM-host Affinity (DRS)

Required rules

Preferential rules
Rule enforcement: 2 options
• Required: DRS/HA will never violate the
rule; event generated if violated manually.
Only advised for enforcing host-based
licensing of ISV apps.
• Preferential: DRS/HA will violate the rule if
necessary for failover or for maintaining
availability

88 Confidential
 Hard Rules

Hard Rules
• DRS will follow the hard rules
• With DPM hosts will get powered on to follow a rule
• If DRS can’t follow,
vCenter will
display an alarm
• Can not be
overwritten by user
• DRS will not generate any recommendations which would violate hard rules
DRS Groups and hard rules with HA
• Hosts will be tagged as “incompatible” in case of “Must Not run…” so HA will
take care of these rules, too

89 Confidential
 Soft Rules

Soft Rules
• DRS will follow a soft rule if possible
• Will allow actions
 User-initiated
 DRS-mandatory
 HA actions
• Rules are applied as long as their application does not impact satisfying
current VM cpu or memory demand
• DRS will report a warning if the rule isn’t followed
• DRS does not produce a move recommendation to follow the rule
• Soft VM/host affinity rules are treated by DRS as "reasonable effort"

90 Confidential
 Grouping Hosts with different capabilities

DRS Groups
Manager
• Defines Groups
• VM groups
• Host groups

91 Confidential
 Managing ISV Licensing

Example
• Customer has 4-node cluster
• Oracle DB and Oracle BEA are charged for every hosts that can run it.
vSphere 4.1 introduces “hard partitioning”
• Both DRS and HA will honour this boundary.

Rest of VMs Oracle DB

DMZ VM Oracle BEA

DMZ LAN
Production LAN

92 Confidential
 Managing ISV Licensing

Hard partitioning
• If a host is in a VM-host must affinity rule, they are considered compatible
hosts, all the others are tagged as incompatible hosts. DRS, DPM and HA are
unable to place the VMs on incompatible hosts.
Due to the incompatible host designation, the mandatory VM-Host is a
feature what can be (undeniably) described as hard partioning. You cannot
place and run a VM on incompatible host
• Oracle has not acknowledged this as hard partitioning.

Sources
• http://frankdenneman.nl/2010/07/vm-to-hosts-affinity-rule/
• http://www.latogalabs.com/2010/07/vsphere-41-hidden-gem-host-affinity-
rules/

93 Confidential
 Example of setting-up: Step 1

In this example, we are adding the “WinXPsp3” VM to the group.

The group name is “Desktop VMs”

94 Confidential
 Example of setting-up: Step 2

Just like we can group VM, we can also group ESX

95 Confidential
 Example of setting-up: Step 3

We have grouped
the VMs in the
cluster into 2

We have grouped
the ESX in the
cluster into 2

96 Confidential
 Example of setting-up: Step 4

This is the screen where we do the

mapping.
• VM Group mapped to Host Group

97 Confidential
 Example of setting-up: Step 5

Mapping is done.
The Cluster Settings
dialog box now
display the new
rules type.

98 Confidential
 HA/ DRS

DRS lists rules

• Switch on or off
• Expand to display
DRS Groups

Rule details
• Rule policy
• Involved Groups

99 Confidential
100 Confidential
 Enhancement for Anti-affinity rules
Now more than 2 VMs in a rule
Each rule can have a
couple of VMs
• Keep them all
together
• Separate them
through cluster
 For each VM
at least
1 host is needed

101

101 Confidential
 DPM Enhancements

Scheduling DPM
• Turning on/off DPM is now a
scheduled task
• DPM can be turned off prior to
business hours in anticipation for
higher resource demands

Disabling DPM
• It brings hosts out of standby
• Eliminates risk of ESX hosts
being stuck in standby mode
while DPM is disabled.
• Ensures that when DPM is
disabled, all hosts are
powered on and ready to
accommodate load increases.

102 Confidential
 DPM Enhancements

103 Confidential
 vMotion

104 Confidential
 vMotion Enhancements

• Significantly decreased the overall migration time (time will vary depending
on workload)
• Increased number of concurrent vMotions:
 ESX host: 4 on a 1 Gbps network and 8 on a 10 Gbps network
 Datastore: 128 (both VMFS and NFS)

• Maintenance mode evacuation time is greatly decreased due to above

improvements
105 Confidential
 vMotion

Re-write of the previous vMotion code

• Sends memory pages bundled together instead of one after the other
 Less network/ TCP/IP overhead
• Destination pre-allocates memory pages
• Multiple senders/ receivers
 Not only a single world responsible for each vMotion thus limit based on host CPU
• Sends list of changed pages instead of bitmaps
Performance improvement
• Throughput improved significantly for single vMotion
 ESX 3.5 – ~1.0Gbps
 ESX 4.0 – ~2.6Gbps
 ESX 4.1 – max 8 Gbps
• Elapsed reduced by 50%+ on 10GigE tests.
Mix of different bandwidth pNICs not supported

106 Confidential
 vMotion

Aggressive Resume
• Destination VM resumes earlier
 Only workload memory pages have been received
 Remaining pages transferred in background
Disk-Backed Operation
• Source host creates a circular buffer file on shared storage
• Destination opens this file and reads out of it
• Works only on VMFS storage
• In case of network failure during transfer vMotion falls back to disk based
transfer
 Works together with aggressive resume feature above

107 Confidential
 Enhanced vMotion Compatibility Improvements
• Preparation for AMD Next Generation without 3DNow!
• Future AMD CPUs may not support 3DNow!
• To prevent vMotion incompatibilities, a new EVC mode is introduced.

108 Confidential
 EVC Improvements

Better handling of powered-on VMs

• vCenter server now uses a live VM's CPU feature set to determine if it can be
migrated into an EVC cluster
• Previously, it relied on the host's CPU features
• A VM could run with a different vCPU than the host it runs on
 I.e. if it was initially started on an older ESX host and vMotioned to the current one
 So the VM is compatible to an older CPU and could possibly be migrated to the EVC
cluster even if the ESX hosts the VM runs on is not compatible

109 Confidential
 Enhanced vMotion Compatibility Improvements
Usability Improvements
• VM's EVC capability: The VMs tab for hosts and clusters now displays the
EVC mode corresponding to the features used by VMs.

• VM Summary: The Summary tab

for a VM lists the EVC mode
corresponding to the features used
by the VM.

110 Confidential
 EVC (3/3)

Earlier Add-Host Error detection

• Host-specific incompatibilities are now displayed prior to the Add-Host work-
flow when adding a host into an EVC cluster
• Up to now this error occurred after all needed steps were done by the
administrator
• Now it’ll warn earlier

111 Confidential
 Licencing
Host-Affinity, Multi-core VM, Licence Reporting Manager

112 Confidential
 Multi-core CPU inside a VM

Click this

113 Confidential
 Multi-core CPU inside a VM

2-core, 4-core, 8 core.

No 3-core, 5 core, 6 core, etc
Type this manually

114 Confidential
 Multi-core CPU inside a VM

How to enable (per VM, not batch)

• Turn off VM. Can not be done online.
• Click Configuration Parameters
• Click Add Row and type cpuid.coresPerSocket in the Name column.
• Type a value (2, 4, or 8) in the Value column.
 The number of virtual CPUs must be divisible by the number of cores per socket.
The coresPerSocket setting must be a power of two.
Notes:
• If enabled, CPU Hot Add is disabled

115 Confidential
 Multi-core CPU inside a VM

Once enabled, it is not readily shown to administrator

This is not shown easily in the UI.
• VM listing in vSphere Client does
not show core
Possible to write scripts
• Iterates per VM
Sample tools
• CPU-Z
• MS SysInternals

116 Confidential
 Customers Can Self-Enforce Per VM License Compliance

When customer use more than they bought

• Alert by vCenter
• But will be able to continue managing additional VMs. So can over use.
• Customers are responsible for purchasing additional licenses and any back-
SNS. So Support & Subscription must be back dated. This is consistent with
current vSphere pricing.

117 Confidential
 Thank You
I’m sure you are tired too 

Confidential
© 2009 VMware Inc. All rights reserved
 Useful references
• http://vsphere-land.com/news/tidbits-on-the-new-vsphere-41-release.html
• http://www.petri.co.il/virtualization.htm
• http://www.petri.co.il/vmware-esxi4-console-secret-commands.htm
• http://www.petri.co.il/vmware-data-recovery-backup-and-restore.htm
• http://www.delltechcenter.com/page/VMware+Tech
• http://www.kendrickcoleman.com/index.php?/Tech-Blog/vm-advanced-iso-free-tools-for-advanced-tasks.html
• http://www.ntpro.nl/blog/archives/1461-Storage-Protocol-Choices-Storage-Best-Practices-for-vSphere.html
• http://www.virtuallyghetto.com/2010/07/script-automate-vaai-configurations-in.html
• http://searchvmware.techtarget.com/tip/0,289483,sid179_gci1516821,00.html
• http://vmware-land.com/esxcfg-help.html
• http://virtualizationreview.com/blogs/everyday-virtualization/2010/07/esxi-hosts-ad-integrated-security-gotcha.aspx
• http://www.MS.com/licensing/about-licensing/client-access-license.aspx#tab=2
• http://www.MSvolumelicensing.com/userights/ProductPage.aspx?pid=348
• http://www.virtuallyghetto.com/2010/07/vsphere-41-is-gift-that-keeps-on-giving.html

119 Confidential
 vSphere Guest API
It provides functions that management agents and other software can use to collect data about the state and
performance of a VM.
• The API provides fast access to resource management information, without the need for authentication.
The Guest API provides read‐only access.
• You can read data using the API, but you cannot send control commands. To issue control commands, use the
vSphere Web Services SDK.
Some information that you can retrieve through the API:
• Amount of memory reserved for the VM.
• Amount of memory being used by the VM.
• Upper limit of memory available to the VM.
• Number of memory shares assigned to the VM.
• Maximum speed to which the VM’s CPU is limited.
• Reserved rate at which the VM is allowed to execute. An idling VM might consume CPU cycles at a much lower
rate.
• Number of CPU shares assigned to the VM.
• Elapsed time since the VM was last powered on or reset.
• CPU time consumed by a particular VM. When combined with other measurements, you can estimate how fast
the VM’s CPUs are running compared to the host CPUs

120 Confidential