Professional Documents
Culture Documents
vSphere 4.0
VMware Confidential
Rev. G
Agenda
Module 0 - Product Overview Module 1 - VI Installation-Upgrade Module 2 - VirtualCenter Module 3 - Storage Module 4 - Networking
vCenter
vCenter
The standard virtual switch that is available in ESX 3.x and 4.x without vNetwork
dvPort Port in a dvSwitch that allows VMs, vnics, VMKernel or Service Console nics. dvPort status is stored in VC Database, so it is persistent across hosts dvPortgroup
Distributed Switch
Distributed Switch: this means that the configuration is centralised to vCenter. All the hosts that belong to a dvSwitch will not need further configuration to be compliant Distributed Switch: the behaviour will still be the same (or consistent) with the vSwitch we are used to deal with: dvPortgroups, as a set of dvPorts (the dv equivalent of Portgroups as a set of ports in a vSwitch) Configuration is inherited from dvSwitch to dvPortgroup (the equivalent of what happens for vSwitch/Portgroup) VMs, Service Console interface (vswif) and VMKernel interfaces can be connected to dvPortgroups as they could be connected to Portgroups in vSwitches Hosts still own 2 configuration contexts, which are therefore not administered centrally via vNetwork:
vCenter
Distributed vSwitch
Control Plane
vSwitch
Data Plane
vSwitch
vSwitch
vSwitch
IO Filter
IO Filter
IO Filter
vSwitch
Port Port Port
Data Plane
Port
IO Filter
Filters (DVN Switch API, or dvFilter) Forwarding (DVN Appliance API, or VSafe-net)
Uplink Abstraction
UPLINK groups allow for abstraction from the physical implementation of each server. Each Physical host can contribute with up to 1 NIC to each Uplink group vCenter will only see the uplink groups when configuring the Distributed Switch, because each host can contribute in a different way (vmnic0,1,2,3,)
vmnic0,1,2,3,?
vCenter
L2 Switch VLAN Segmentation 802.1Q Tagging NIC Teaming TX Rate Limiting RX Rate Limiting Unified management interface PVLAN 3rd Party Virtual Switch Support
10
11
12
vC Extension
Control Plane
ESX 4
Control Plane Appliance
DataPlane Agent Data Plane
ESX 4 ESX 4
Distributed vSwitch
vSwitch
vSwitch
13
14
15
Uplinks
16
17
18
19
20
21
22
Port Binding
Static Binding (Default): means that the dvPort will be assigned to the VM at configuration time. Once all the ports are booked by VMs, it will not be possible to connect any more VM, independently from the fact that the connected VMs are powered up or not, and an error message will be displayed Dynamic Binding: means that the dvPort will be assigned at the moment of powering the VM up. This option allows for over committing the number of dvPorts. Ephemeral Ports or No Binding: this behaviour has been introduced to resemble the behaviour in the standard vSwitch. If you select this option, the number of ports will be automatically set to 0, and the Portgroup will allocate one port for each connected VM, up to the maximum number of ports available in the Switch.
vSphere 4- Mod 4 - Slide 23
24
25
26
27
28
29
30
From the screenshot on the right, you can see how the Active/Standby status is applied to each uplink group (dvUplink1 and 2 in this case), and not to the vmnics directly, as it used to be with standard vSwitches
31
32
33
34
35
36
37
38
39
40
41
vSwitch0
vmk0 1
dvPG0
Create a DS with as many Uplink groups as Physical NICs connected to the Standard Switches Create in the DS as many Portgroups as you already have in the SS Assign Uplinks to each Portgroup in the DS
2. 3.
vm1 vm2
2
Uplink4 3 vSwitch1 DS dvPG1
4.
Break each teaming and transfer one NIC from each vSwitch to a corresponding Uplink group
Migrate the Virtual Adapters and the Virtual Machines to the appropriate Portgroups Transfer the remaining uplinks to the Uplink groups associated with the appropriate Portgroups Remove the Standard Switches and their Portgroups
5. 6.
7.
42
Lab Exercise
Lab 1: vNetwork Distributed Switch
44
45
What is a Private VLAN? VLAN is a mechanism to divide a broadcast domain into several logical broadcast domains
Private VLAN is an extension to the VLAN standard, already available in several (most recent) physical switches. What it does is add a further segmentation of the logical broadcast domain, to create Private groups
Furthermore, because it divides a VLAN (which will be called Primary PVLAN) into one or more groups (called Secondary PVLANs), this means that all the Secondary PVLANs exist only within the Primary VLAN. Private because, depending upon the type of the groups involved, hosts will not be able to communicate each other, even if they belong to the same group. Each Secondary PVLAN has an associated VLAN ID, and the physical switch will associate the behaviour (Isolated, Community or Promiscuous) depending on the VLAN ID found in each packet.
vSphere 4- Mod 4 - Slide 46
17
Community
Host 1
155
Host 2
17 5
Host 4
Host 3
Host 5 Host 6
Three types of Secondary PVLANs: Promiscuous A node attached to a port in a promiscuous secondary PVLAN may send and receive packets to any node in any others secondary VLAN associated to the same primary. Routers are typically attached to promiscuous ports. Isolated A node attached to a port in an isolated secondary PVLAN may only send to and receive packets from the promiscuous PVLAN. Community A node attached to a port in a community secondary PVLAN may send to and receive packets from other ports in the same secondary PVLAN, as well as send to and receive packets from the promiscuous PVLAN.
47
Secondary
5 155 17
Type
Promiscuous Isolated Community
VLAN 5
PVLAN 5 (Promiscuous )
PVLAN 17 (Community)
48
49
CONS:
A a lot of subnets of the /30 type, with waste of IP addresses (50%) Consequently, lot of routes, which are difficult to maintain and change Complex and expensive gateway (firewall) rules
Available VLANs are 4095*, but switches allow much less, about 1000
Too complex/expensive to maintain One VLAN per VM, with one VM acting as transparent/software bridge with firewall, thus on the same subnet Can be implemented inside ESX 3.x Even more complexity/cost PVLAN
vSphere 4- Mod 4 - Slide 50
Internet
Gateway
Example: Hosting company: Many different customers that should not be able to see each other Possible solution: One VLAN per customer, but: Creating a VLAN for each customer is expensive: One subnet per customer is required, gateway maintenance is a nightmare If a customer grows in size, subnets might have to be changed (for example /30 to /29) Physical switches can handle a limited amount of VLANs per switch (less than 4000)
Several /30 Subnets
51
PVLANs Single Subnet Gateway in the promisc PVLAN Each Customer in Isolated PVLAN Community PVLAN if Customer expands
Gateway
52
Community
All the machines connected to a Community PVLAN Portgroup can send to and receive from any other machine on the same Community or Promiscuos PVLAN associated with the same primary VLAN Isolated
Each machine connected to an Isolated PVLAN Portgroup can send to or receive from only machines on the Promiscuous PVLAN associated to the same primary VLAN
vSphere 4- Mod 4 - Slide 53
Secondary
5 155 17
Type
Promiscuous Isolated Community
54
Because this way the mac address is associated to the single port.
55
Switch ports that see the same mac address through different VLAN tags
PVLAN logic detects that the destination is Isolated so act as if the tag were 155
dvSwitch
5 5 5
Promisc Isolated
Arp Reply Tag: none Arp Reply Tag: 155
vSphere 4- Mod 4 - Slide
VM 1 cant talk to any VM in PVLAN 155 in PVLAN 17 VM 1 can talk to VMs in PVLAN 5 in Virtual Switches Physical Switch VM 1 can talk to VM 2 and 3 only if the physical switch is configured to handle PVLAN 155. If the Physical switch allows VLAN 155, the isolation might be compromised.
5 5 5
Physical
dvSwitch
VM 1
155
155 5
17
5 5
VM 2 VM 6 VM 3
VM 5 VM 4
57
VM 7 cant talk to any VM in PVLAN 155 VM 7 can talk to VMs in PVLAN 17 in PVLAN 5 in Virtual Switches Physical Switch VM 7 can talk to VM 2 and 3 only if the physical switch is configured to handle PVLAN 17. If the Physical switch allows VLAN 17, the isolation might be compromised.
5 5 5
Physical
dvSwitch
155
17 5
17
VM 7
5 5
VM 2 VM 6 VM 3
VM 5 VM 4
58
On the Primary Tab, add the VLAN that will be used outside the PVLAN domain, and select it On the Secondary Tab, create the PVLANs of the desired type. There can be only one Promiscuous PVLAN and is created automatically for you. Beware: before deleting any primary/secondary PVLAN, make sure that they are not in use, or the operation will not be performed.
59
Lab Exercise
Lab 2: Using PVLANs
60
Break
61
62
IPv6
IPv6 Concepts VI4 and IPv6 New TCP/IP Stack GuestOS and IPv6
63
IPv6 Concepts - 1
IP Next Generation (v4 was officialised in 1981) Addresses are 128-bits long
64
IPv6 Concepts - 2
No more IP broadcasts, but advanced multicast IPv6 has autoconf capabilities, and via multicast can discover routers and receive the configuration from them. There is also an IPv6 version of DHCP. DNS can serve IPv6 entries, even over IPv4 connections (or vice versa).
IPv6 can be tunnelled over IPv4, but they cant be mixed (you cant access an IPv6 host via an IPv4 network, only across an IPv4 network via tunnels.
65
; <<>> DiG 9.5.0-P2 <<>> www.ipv6.org AAAA ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57681 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 4, ADDITIONAL: 0
;; QUESTION SECTION: ;www.ipv6.org. ;; ANSWER SECTION: www.ipv6.org. shake.stacken.kth.se. ;; AUTHORITY SECTION: stacken.kth.se. stacken.kth.se. stacken.kth.se. stacken.kth.se. ;; ;; ;; ;;
IN 3600 3600 IN IN
IN IN IN IN
NS NS NS NS
Query time: 671 msec SERVER: 10.21.64.212#53(10.21.64.212) WHEN: Tue Nov 4 16:21:06 2008 MSG SIZE rcvd: 174
66
vCenter will display correctly IPv6 addresses for Service Console, VMKernel and VMs as reported by the tools
67
68
IPv6 Address Dialog box: The box where you can enter the IPv6 address is free-form. There is no more the concept of subnet mask, but subnet prefix, which is the number of bits that constitute the prefix (Similar to CIDR notation for IPv4)
vSphere 4- Mod 4 - Slide 69
IPv4 module is loaded by default Based on FreeBSD 6.1 Improved performance and scalability due to locking and threading improvements (more CPUs can be used) If IPv6 is enabled for the VMKernel, it will look like this:
# vmkload_mod -l Name tcpip2v6
R/O Addr 0x4180225fd000 Length 0xbd000 R/W Addr 0x417fe3676f80 Length 0x37000 ID Loaded 47 Yes
Note: esxcfg-module -l is equivalent to vmkload_mod -l, and is available also in the vi-cli.
70
By using the vi-cli (also available in the vMA), with the command:
esxcfg-module l
$ esxcfg-module -l --server esxi.vmware.com --username root --password secret Name ID Loaded tcpip2 45 Yes
$ esxcfg-module -l --server esxi.vmware.com --username root --password secret Name ID Loaded tcpip2v6 45 Yes
Since there is no service console here, the lsmod part is not necessary.
71
In Linux, IPv6 is supported since 2.4 but the implementation is not fully compliant until 2.6 versions
In Windows, 2003 SP1 and XP SP2 have the infrastructure for IPv6, even though some components of the system and applications are not IPv6-ready. (For 2003 check http://technet.microsoft.com/enus/library/cc776103.aspx) Vista and 2008 fully support IPv6
72
73
IPV6_AUTOCONF specifies whether IPV6 advertising should be used to configure NICs IPV6_DEFAULTGW can have a %eth0 appended at the end, thus overriding IPV6_DEFAULTDEV
/etc/sysconfig/network-scripts/ifcfg-eth0 contains the information to configure both IPv4 and IPv6, for example:
DEVICE=eth0 ONBOOT=yes BOOTPROTO=static BROADCAST=172.16.5.255 NETMASK=255.255.255.0 DHCPV6C=no IPADDR=172.16.5.99 IPV6ADDR=fec0::d/112 IPV6INIT=yes IPV6_AUTOCONF=no
IPV6_ADDR contains also the prefix size (similar to IPv4 Netmask, in CIDR format)
74
IPv6 commands will generally have a -6 option or a 6 at the end to distinguish from the IPv4 equivalents ip
ip -6 address add fec0::5/112 dev eth0 ip -6 route add default via fec0::1
ping
ping6 fec0::1
tracepath
tracepath6 fec0::1
traceroute
traceroute6 fec0::1
iptables
ip6tables
75
netsh has several dump commands you can use to get information netsh interface ipv6 dump netsh interface dump
76
Lab Exercise
Lab 4: IPv6
77
78
VMXNET Generation 3
New state of the art Virtual Network Adapter
Also known as Advanced VMXNET Based on Enhanced VMXNET introduced in ESX 3.5 Introduces new features: IEEE 802.1Q VLAN Tagging. No more need for e1000 in such a case VLAN Tagging and Tag removal offloading Only one VLAN per NIC for Windows TCP Segmentation Offloading for IPv4 and IPv6 TCP and UPD Checksum Offloading for IPv4 and IPv6 MSI (Messaged Signalled Interrupt) and MSI-X support (subject to guest kernel support) Receive Side Scaling (supported in Windows Vista, 2008 and any other system using NDIS 6.x)
vSphere 4- Mod 4 - Slide 79
VMXNET Generation3
No Record/Replay support Supported Guest OSes (both 32-bit and 64-bit versions):
80
Lab Exercise
Lab 5: VMXNET Generation 3
81
82
VMDirectPath I/O - 1
VMDirectPath I/O is a mechanism by which VMs are allowed to directly access a physical device using the native driver in the GuestOS. Each Device will be accessible by one single VM. Main use cases for this feature are I/O devices that may have high performance/low-latency/CPU efficiency requirements VMDirectPath I/O (Also known as Fixed Passthrough) is fully supported for networking I/O devices with the Intel 82598 10 Gigabit Ethernet Controller and Broadcom 57710 10 Gigabit Ethernet Controller experimentally supported for storage I/O devices with the QLogic QLA25xx 8Gb Fibre Channel and the LSI 3442e-R and 3801e (1068 chip based) 3Gb SAS adapters.
83
VMDirectPath I/O - 2
Support will be limited to Intel and AMD CPUs with EPT/NPT/RVI and IOMMU (VT-d for Intel) support The following features are unavailable: VM cant be VMotion-ed (Uniform Pass Through will allow VMotion, but it is not available in vSphere 4.0) Therefore, DRS (limited availability The virtual machine can be part of a cluster, but cannot migrate across hosts) Hot add/remove of virtual devices Suspend and Resume Record and Replay Fault Tolerance High Availability Memory Overcommitment and Page Sharing
84
85
86
You will need to reboot for the devices to become ready (Green)
vSphere 4- Mod 4 - Slide 87
Note: the configuration changes will go into /etc/vmware/esx.conf. In the case above, the PCI slot where the device was connected is 00:0b:0, so it will be:
/device/000:11.0/owner = "passthru (0b is 11 in decimal)
vSphere 4- Mod 4 - Slide 88
89
90
91
92
Lab Exercise
Lab 6: VMDirectPath I/O
94
95
The Virtual Machine Communication Interface (VMCI) is an infrastructure that provides fast and efficient communication between a virtual machine and the host operating system and between two or more virtual machines on the same host.
The VMCI SDK facilitates development of applications that use the VMCI infrastructure. Without VMCI, virtual machines communicate with the host using the network layer.
Using the network layer adds overhead to the communication. With VMCI communication overhead is minimal and different tasks that require that communication can be optimized.
96
num is a positive integer that is unique for each virtual machine on your host. That is, for any virtual machine, you can choose a number (1, 2, 3, etc.) but two virtual machines must not have the same number as their vmci0.id.
You also need the VMCI component of the VMware Tools to be installed inside the VM
97
98
10
gbps
0 128 256 512 1024 2048 4096 8192 16384 32768 65536
Message Size
99
Break
100
101
102
VMX changes
DVS
ethernet1.dvs.switchId = "7a f2 34 50 21 55 6c 70-a4 b1 10 f1 3f 9d 2c c1" ethernet1.dvs.portId = "1423" ethernet1.dvs.connectionId = "419447540" ethernet1.dvs.portgroupId = "dvportgroup-302
VMXNET3
ethernet0.virtualDev = "vmxnet3
103
net-dvs output
switch 7a f2 34 50 21 55 6c 70-a4 b1 10 f1 3f 9d 2c c1 (etherswitch) Global properties: Uplink Identifiers com.vmware.common.alias = dvSwitch com.vmware.common.uplinkPorts = dvSwitch identifier Uplink1,Uplink2,Uplink3,Uplink4 com.vmware.common.host.uplinkPorts = dvSwitch DVPorts used for Uplink 5,6,7,8 Name com.vmware.etherswitch.pvlanMap = (11, 11) - Promiscuous PVLAN map dvPortgroup Map, associating (11, 12) Community vCenter dvPortgroup names and dvPortgroup labesl (11, 13) - Isolated MTU (68, 68) - Promiscuous 1500 = 0x5DC (68, 681) - Isolated (beware of (68, 682) - Community endian-ness) CDP Enabled 0/1 com.vmware.etherswitch.mtu = 0xdc. 5. 0. 0 com.vmware.etherswitch.cdp = 0x 0. 1 com.vmware.common.pgmap =vSwitch-DVUplinks-211:dvportgroup212,PVLAN-11-I:dvportgroup-239,PVLAN-11-C:dvportgroup-240,VGT:dvportgroup241,PVLAN-11-P:dvportgroup-242,VLAN68:dvportgroup-243,PVLAN-68-I:dvportgroup244,PVLAN-86-C:dvportgroup-245,PVLAN-68-P:dvportgroup -246,Ghost:dvportgroup-299,dvPortGroup:dvportgroup-300,VLAN64:dvportgroup-302 Host properties: com.vmware.common.host.portset = DvsPortset-1
105
net-dvs output
port 5 com.vmware.common.port.alias = Uplink1 com.vmware.common.port.connectid = 1912494964 com.vmware.common.port.portgroupid = dvportgroup-212 com.vmware.common.port.block = false com.vmware.etherswitch.port.teaming = load balance = source virtual port id link selection: link state up; link speed>=10Mbps; link behavior: notify switch; reverse filter; best effort on failure; shotgun on failure; active: standby: com.vmware.etherswitch.port.security = 0x 1. 0. 0. 0 com.vmware.etherswitch.port.vlan = Guest VLAN tagging ranges: 1-4094 com.vmware.common.port.statistics: pktsInUnicast = 1699111 bytesInUnicast = 865718684 pktsInMulticast = 2204789 bytesInMulticast = 580474616 pktsInBroadcast = 7441346 bytesInBroadcast = 623725320 pktsOutUnicast = 1091384 bytesOutUnicast = 783242007 pktsOutMulticast = 34 bytesOutMulticast = 2744 pktsOutBroadcast = 2069749 bytesOutBroadcast = 179071956 pktsInDropped = 159 pktsOutDropped = 0 pktsInException = 1285 pktsOutException = 0 com.vmware.common.port.volatile.vlan = VLAN 0 ranges: 1-4094 com.vmware.common.port.volatile.status:inUse linkUp portID = 0x2000002
106
net-dvs output
port 519 com.vmware.common.port.alias = com.vmware.common.port.connectid = 1502730467 com.vmware.common.port.portgroupid = dvportgroup-241 com.vmware.common.port.block = false com.vmware.etherswitch.port.teaming = load balance = source virtual port id link selection: link state up; link speed>=10Mbps; link behavior: notify switch; reverse filter; best effort on failure; shotgun on failure; active: Uplink1 Uplink2 Uplink3 Uplink4 standby: com.vmware.etherswitch.port.security = 0x 0. 0. 0. 0 com.vmware.etherswitch.port.vlan = Guest VLAN tagging ranges: 11-14 64-72 com.vmware.common.port.volatile.persist = /vmfs/volumes/f1c540c6-3bd757e8/.dvsData/7a f2 34 50 21 55 6c 70-a4 b1 10 f1 3f 9d 2c c1/519 com.vmware.common.port.volatile.vlan = VLAN 0 ranges: 11-14 64-72 com.vmware.common.port.statistics: pktsInUnicast = 3972 bytesInUnicast = 571094 pktsInMulticast = 27 bytesInMulticast = 2166 pktsInBroadcast = 17 bytesInBroadcast = 2712 pktsOutUnicast = 6499 bytesOutUnicast = 7405784 pktsOutMulticast = 2488 bytesOutMulticast = 664816 pktsOutBroadcast = 1103380 bytesOutBroadcast = 95151238 pktsInDropped = 0 pktsOutDropped = 0 pktsInException = 503 pktsOutException = 0 com.vmware.common.port.volatile.status:inUse linkUp portID = 0x200000d
107
net-dvs notes
Launch with /usr/lib/vmware/bin/net-dvs Output collected by vm-support Not Available for ESXi unless you connect directly via SSH (Not supported) DVS information is cached in /etc/vmware/dvsdata.db Binary file Collected by vm-support Can be used to produce net-dvs output from any linux host (for example scripts server) with the net-dvs f [FILE] command DVS Port information is stored in a shared VMFS volume root, under .dvsData/, net-dvs output will indicate the exact location. This can be useful to quickly locate which ports are still accessing a given DSwitch References to the DVS are also on /etc/vmware/esx.conf VMKernel ports
vSphere 4- Mod 4 - Slide 108
109
esxcfg-vswitch
#esxcfg-vswitch -l Switch Name Num Ports vSwitch0 32 PortGroup Name Switch Name vSwitch1 Used Ports 2 Configured Ports 32 Uplinks MTU 1500 Uplinks vmnic1 MTU 1500 Uplinks vmnic0
VLAN ID
Used Ports
Num Ports 64
Used Ports 3
PortGroup Name VM Network DVS Name dvSwitch DVPort ID 5 6 7 8 391 390 1422 1419 1423 519 1420
VLAN ID 0
Used Ports 1
Num Ports 64
Used Ports 6
Uplinks vmnic3,vmnic2
In Use 1 1 0 0 0 0 1 1 1 0 0
vmk0 vswif1
110
esxcfg-vswif
Create a new vswif Same syntax as ESX 3.x
esxcfg-vswif -a vswif1 -i 10.21.64.25 -n 255.255.252.0 -p Service Console
Output of esxcfg-vswif l
Name vswif1 vswif1 vswif1 Port Group/DVPort 1419 1419 1419 IP Family IPv4 IPv6 IPv6 IP Address 10.21.64.25 fec0::4 fe80::250:56ff:fe4f:cba Netmask 255.255.252.0 112 64 Broadcast 10.21.67.255 Enabled true true true TYPE STATIC STATIC STATIC
111
esxcfg-vmknic
Add a vmknic on a vSwitch
esxcfg-vmknic a -i 10.21.66.25 -n 255.255.252.0 p VMKernel Network
Output of esxcfg-vmknic -l
Interface Port Group/DVPort IP Family IP Address Broadcast MAC Address MTU TSO MSS Enabled Type vmk1 1421 IPv4 10.21.66.25 255.255.252.0 10.21.67.255 00:50:56:75:79:ae 1500 65536 vmk1 1421 IPv6 fe80::250:56ff:fe75:79ae 00:50:56:75:79:ae 1500 65536 true STATIC vmk1 1421 IPv6 fec0::5 00:50:56:75:79:ae 1500 65536 true STATIC Netmask
true STATIC 64 112
112
esxcfg-route
Add an IPv6 default gateway (all the other operations are the same as 3.5)
esxcfg-route -f V6 -a default fec0::1
Netmask 0 64 112 32 32
Gateway fec0::1 Local Subnet Local Subnet Local Subnet Local Subnet
113
Troubleshooting PVLANs
Key concepts to keep in mind when troubleshooting PVLANs: Packets in PVLANs travel tagged as if they were in a VLAN with ID as the Secondary ID, there is no encapsulation. This is valid for both virtual and physical switches Physical switches need to be configured to forward packets in such VLAN IDs between source and destination Consider PVLAN as a particular case of VST, so: Physical switch to ESX should be trunking Physical switches should be connected via trunks Unless they are not PVLAN aware, in which case the trunk should be a PVLAN trunk if you are using Isolated PVLANs Physical hosts should be connected to a PVLAN port
VTP (Vlan Trunking Protocol) has to be in transparent mode in the physical switch, because PVLANs are defined locally on the single physical switch
vSphere 4- Mod 4 - Slide 114
Troubleshooting PVLANs
Troubleshooting hints Make sure that the physical and virtual switch configuration matches:
Physical switch port is trunking for all the primary and secondary PVLAN IDs
Compare the PVLAN maps in physical and virtual switch In Cisco switches, you can use the commands:
show running-configuration
show interface private-vlan mapping show interface [interface-id] switchport
115
Similarly, create the secondary PVLAN (ex. VLAN 13, Isolated, 12, Community)
(config)# vlan (vlan-config)# (config)# vlan (vlan-config)# 13 private-vlan isolated 12 private-vlan community
Bind switch ports to the PVLANs (1/10 Isolated, 1/11 Community and 1/1 promisc):
(config)# interface Fastethernet 1/10 (config-if)# switchport mode private-vlan host (config-if)# switchport private-vlan host-association 11 12 (config)# interface Fastethernet 1/11 (config-if)# switchport mode private-vlan host (config-if)# switchport private-vlan host-association 11 13 (config)# interface Fastethernet 1/1 (config-if)# switchport mode private-vlan promiscuous (config-if)# switchport private-vlan mapping 11 12,13
116
Ensure that your uplinkportprofile1 includes the VLAN that is configured on your VMs port profile.
# show port-profile name uplinkportprofile1
To isolate how far the traffic gets, do tcpdump inside the VMs, cb print ingress on the DP, and debug ip packets detail on the upstream Cisco switch
117
esxcfg-firewall - 1
New feature (soon available also in 3.5) of filtering connections per host/port, with the option:
--ipruleAdd <host,cport,tcp|udp,REJECT|DROP|ACCEPT,name>
As you might already know from ESX 3.x, list firewall rules with esxcfgfirewall q and be careful, because -l will reload the firewall rules instead, overwriting the possible root cause of your investigation. There is no mechanism to temporarily stop the firewall like in ESX 3.5 using service firewall stop|start because the service firewall stop will not do anything but print the following:
firewall can't be stopped. To disable the firewall run, esxcfg-firewall --allowIncoming allowOutgoing
118
esxcfg-firewall - 2
But keep in mind that: esxcfg-firewall --allowIncoming allowOutgoing modifies the firewall configuration, so to return to the previous configuration you need to use esxcfg-firewall --blockIncoming --blockOutgoing, because esxcfg-firewall -l wont. If you use allowIncoming and allowOutgoing, previously defined IP Rules will still be applied
119
esxcfg-firewall - 3
So what can we do for temporarily disabling the firewall for troubleshooting? Remember to save the actual configuration before doing anything else! Otherwise you might not be able to identify the root cause. Save the output of iptables -L or better of iptables-save to a file. You can use iptables -F or iptables-save, and then reload the firewall with esxcfg-firewall l, when the troubleshooting is done.
120
esxcfg-firewall - 3
With iptables F youll flush all the rules. Keep in mind that usually the default policy is to drop connections, and the rules are allowing you in. This means that before flushing the rules, you should make sure that at least the INPUT chain has default set to ALLOW, with iptables P INPUT ALLOW, or youll lock yourself out. With iptables-save>file you can save to a file the rules, then edit the files so that you remove all the rules and the chains, edit the policy to be ALLOW, review what youve done, and then apply your changes with iptablesrestore<file. For example: *filter :INPUT ACCEPT [4495370:1545008248] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [3029364:951838897] COMMIT
121
Maximums - 1
Maximums Switches per VC Switches per ESX host Port groups per ESX host Port groups per switch Ports per host Uplinks per host Ports per switch Uplinks per virtual switch Max number of hosts per switch VLANs/Private VLANs
4096
32 1016 32 NA
4096
32 1016 32 NA
4096
32 8000 32 300
122
Maximums - 2
Physical NIC Type Max Number of ports per ESX Host 32 16 32* 4 32* 4 16 4 4
Type
VMKernel Service Console
tg3 (Broadcom 1GigE) bnx2 (Broadcom 1GigE) e1000e (Intel 1GigE PCIe) s2io (Neterion 10GigE) e1000 (Intel PCIx) nx_nic (Netxen 10GigE) Igb (Intel Zoar) bnx2x (10GigE Broadcom) igbe (Intel 10GigE Oplin)
(*) If the Hardware supports them.
Hardware Version 4 7
123
Known Issues
VMDirectPathI/O requires GuestOS support. For example, Oplin NIC in passthrough mode does not perform well with SLES10 in VGT mode.
IPv6 default gateway might not be effective: you might want to use static routes for the specific destination
Removing IPv4 default gateway might cause IPv6 default gateway to fail, especially if the gateway does not do IPv6 advertisment. Configuring a NIC with neither ant static IP (v4 or v6) nor any dynamic configuration (no DHCP not IPv6 autoconf), after reboot you will have to remove it and add it again to be able to reconfigure it.
124
Recovering
Find out the uplink port for the NIC you want to use esxcfg-vswitch -l Remove the Uplink from the DVS esxcfg-vswitch -Q vmnic1 -V 5 dvSwitch Create a new LifeSaver Standard vSwitch esxcfg-vswitch a LifeSaver Give the LifeSaver Standard vSwitch a portgroup and the uplink esxcfg-vswitch A SOSC LifeSaver esxcfg-vswitch L vmnic1 LifeSaver Move the vSwif 0 to the LifeSaver vSwtich esxcfg-vswif d vswif0 esxcfg-vswif a i DHCP p SOSC vswif0 Use vCenter to fix all via GUI and then cleanup, otherwise: esxcfg-vswitch -P vmnic1 -V 5 dvSwitch
125
Questions?