Recovering from a management portgroup VLAN misconfiguration on a vNetwork Distributed Switch (vDS)

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• There is a misconfiguration of the management network on a vNetwork Distributed Switch (vDS) resulting in the host being unmanageable.
• You have misconfigured the VLAN ID on the management dvPortGroup in VMware vCenter Server.
• When the updated configuration is pushed to the hosts, they disconnect from vCenter Server.
• You cannot modify the VLAN ID on the dvPortGroup settings as it requires vCenter Server and the host cannot connect to the network.

Environment

VMware ESXi 4.0.x Installable
VMware ESX 4.0.x
VMware ESX 4.1.x
VMware ESXi 4.1.x Embedded
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 5.0
VMware ESXi 4.0.x Embedded

Cause

Resolution

Check to see if you can access any of other vmkernel ports (such as vMotion, iSCSI, NFS, or FT) on the affected ESX/ESXi host using a ssh client.

Note: To use the ssh client on an ESXi 5.0 host you will need to open the firewall rule to allow the ssh client to work. For further/related information, see Creating Custom Firewall Rules in ESXi 5.0 (2008226).

Note: For ESXi hosts it may not be possible to route to the host through the default gateway therefore you may need to use another ESX/ESXi host which is on the same network.

If you cannot access the host from any other vmkernel port then you need recreate the management network vmkernel port on a standard vSwitch.

The procedure for getting the management network from a vDS to a standard vSwitch:

1. First ensure that any misconfiguration of the vDS has been corrected in vCenter to ensure the misconfiguration is not pushed back to the host after correcting the issue with the management network.
2. Check existing ports for NICs.
   
   # esxcfg-vswitch -l
   
   DVS Name Num Ports Used Ports Configured Ports MTU Uplinks
   dvSwitch 256 6 256 1500 vmnic1,vmnic0
   
   DVPort ID In Use Client
   2 1 vmnic0
   3 1 vmnic1
   101 1 vmk0
   391 1 vmk1
   135 1 vmk2
   263 0
   
   
3. Remove one of the network cards which is configured as an uplink to the vDS, such as vmnic0:
   
   # esxcfg-vswitch -Q vmnic0 -V 2 dvSwitch
   
   
4. Confirm that the network card has been removed from the vDS
   
   # esxcfg-vswitch -l
   
   DVS Name Num Ports Used Ports Configured Ports MTU Uplinks
   dvSwitch 256 6 256 1500 vmnic1
   
   DVPort ID In Use Client
   2 0
   3 1 vmnic1
   101 1 vmk0
   391 1 vmk1
   135 1 vmk2
   263 0
   
   We can see that vmnic0 has been removed from port 2
   
   
5. Create a new standard vSwitch, such as vSwitch0:
   
   # esxcfg-vswitch -a vSwitch0
   
   
6. Create a portgroup on the standard vSwitch where we can create a new vmkernel port
   
   # esxcfg-vswitch -A "VMkernel" vSwitch0
   
   
7. Add the network card that we removed from the vDS (vmnic0 in the above example) as an uplink to the new vSwitch we created, vSwitch0 in our case.
   
   # esxcfg-vswitch -L vmnic0 vSwitch0
   
   
8. List out the current vmkernel ports configuration for the host (For ESX hosts see the note and the end of this procedure):
   
   # esxcfg-vmknic -l
   
   Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
   vmk0 101 IPv4 192.168.0.3 255.255.255.0 192.168.0.255 xx:xx:xx:xx:xx:xx 1500 65535 true STATIC
   vmk1 391 IPv4 xx.xx.xx.xx 255.255.255.0 xx.xx.xx.xx xx:xx:xx:xx:xx:xx 1500 65535 true STATIC
   vmk2 135 IPv4 xx.xx.xx.xx 255.255.255.0 xx.xx.xx.xx xx:xx:xx:xx:xx:xx 1500 65535 true STATIC
   
   
9. Now check to see which vDS port that the management vmkernel port is using, vmk0 is the management vmkernel port in our case
   
   # esxcfg-vswitch -l
   
   Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
   vSwitch0 128 2 128 1500 vmnic0
   
   PortGroup Name VLAN ID Used Ports Uplinks
   VMkernel 0 0 vmnic0
   
   
   DVS Name Num Ports Used Ports Configured Ports MTU Uplinks
   dvSwitch 256 5 256 1500 vmnic1
   
   DVPort ID In Use Client
   2 0
   3 1 vmnic1
   101 1 vmk0
   391 1 vmk1
   135 1 vmk2
   263 0
   
   We can see from above that the vDS port that vmk0 is using is port 101
   
   
10. Delete the management vmkernel port from the vDS, vmk0 is the management port in our case, and its port on the vDS is 101.
    
    # esxcfg-vmknic -d -s "dvSwitch" -v 101 101
    
    
11. Recreate the management vmkernel port on the standard vSwitch:
    
    # esxcfg-vmknic -a -i 192.168.0.3 -n 255.255.255.0 VMkernel
    
    Where the IP address and Netmask were obtained from step 8, and the portgroup VMkernel was created in step 6

Note: The above procedure is suitable for recreating the management vmkernel port on an ESXi host. For an ESX host with a Service Console vswif interface replace the esxcfg-vmknic with appropriate esxcfg-vswif commands.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box
Creating custom firewall rules in VMware ESXi 5.x