ESXi host becomes "Not Responding" after removing a vmnic from a Distributed Switch during decommissioning

Products

• VMware vSphere ESXi

Issue/Introduction

• While decommissioning an ESXi host from vCenter, removing a vmnic from the vSphere Distributed Switch (vDS) causes the host to enter a Not Responding state in vCenter.
• The Management VMkernel (vmk0) is connected to a distributed switch.
• Network connectivity to the host is lost immediately after removing a vmnic.

Environment

VMware ESXi 8.x

VMware vCenter Server 8.x

Cause

• The issue occurs when the host has two or more physical NICs (vmnics) configured, but only one of them provides management network connectivity.
• When the active vmnic with management connectivity was removed from the distributed switch, vmk0 attempted to fail over to the other vmnic, which did not have management network access.
• This failover caused the host to lose management connectivity and become Not Responding in vCenter.

Resolution

Note : Ensure that all vmnics configured for ESXi management have identical network settings and VLAN configurations so that management traffic can successfully fail over if one vmnic is removed.

In scenarios where only one working vmnic is available, the following steps should be performed to restore connectivity:

• Accessed the host console (DCUI) (i.e. ILO/IDRAC/IPMI).
• Log in and go to troubleshooting options -> Enable ESXi Shell -> Press Alt + F1 to enter shell.
  
  
• Create a standard switch and a port group with the required VLAN for VMK0 from the ESXi shell.
  • esxcli network vswitch standard add -v <new_vSwitch_name>
  • esxcli network vswitch standard portgroup add -p <new_portgroup_name> -v <new_vSwitch_name>
  • esxcfg-vswitch -p <portgroup_name> -v <VLAN_ID> <vSwitch_name> (Note : Use the same names mentioned in the above step)
    
    
• Remove and re-create vmk0 on the newly created standard switch.
  • esxcli network ip interface remove -i vmk0
  • esxcli network ip interface add -i vmk0 -p <portgroup_name>
  • esxcli network ip interface ipv4 set -i vmk0 --ipv4="#.#.#.#" --netmask="#.#.#.#" --type=static
    • Example : esxcli network ip interface ipv4 set -i vmk0 --ipv4=10.10.10.10 --netmask=255.255.255.0 --type=static
      
      
• Configure the default gateway for the management network.
  • esxcfg-route -a default <gateway_ip>
    
    
• Remove the working vmnic from the distributed switch and attached it to the new standard switch.
  • Remove uplink from DVS steps mentioned under Resolution step1 of this article
  • Add uplink to standard switch step mentioned under Resolution step 2c of this article
    
• Verify connectivity by pinging the gateway from the host. If the correct vmnic and VLAN are configured, the host should be reachable and reconnected in vCenter.

Decommissioning the Host from vCenter

• To safely decommission the host after restoring connectivity:
  • Remove the host from all associated distributed switches.
    Note : Migrate any remaining VMkernel adapters (e.g., vMotion, FT etc) to a standard switch before removal.
    
    
    • Related Articles :
      • Migrate a VMkernel Adapter on a Host to a vSphere Standard Switch 
      • Remove Hosts from a vSphere Distributed Switch
        
        
  • Disconnect the host from vCenter.
  • Remove the host from inventory once it shows as disconnected.

Additional Information

How to Remove an ESXi Host from Your vCenter Server Instance