Host experiences transient ICMP connectivity during reboot followed by complete network isolation after migrating to LACP uplink.

Products

VMware vSphere ESXi VMware vCenter Server

Issue/Introduction

In VMware vSphere ESXi environments, migrating host uplinks to a vSphere Distributed Switch (vDS) Link Aggregation Group (LAG) may initially complete without packet loss, creating a false positive of a healthy LACP configuration. However, upon the subsequent host reboot, the ESXi host will experience temporary ICMP reachability (e.g., ICMP ping success) before transitioning into complete and consistent network isolation. This transient connectivity window varies depending entirely on the time required for the userworld (hostd) to initialize during the boot sequence; it is completely unrelated to standard LACP protocol timers.

Observed Symptoms and Environmental configuration that leads to the observed issue are:

The vDS is configured for dynamic LACP (lag), while the upstream physical switch is configured for Static EtherChannel (Mode ON).
Initial configuration of uplinks to the vDS LAG succeeds without packet loss, creating a false positive impression of LACP working as expected.
Post-reboot of ESXi hosts, pings to management and other vmkernel interfaces (vmk#) are successful immediately after the physical adapters' drivers initialize.
Following the hostd initialization delay(duration can vary ~60 sec), network communication drops entirely, transitioning directly to consistent connection timeouts.
No preceding packet loss or physical link-flap alerts occur on the physical switch fabric.
The physical switch ports remain in an Up/Up operational state throughout the failure transition, confirming the isolation occurs entirely at the software-defined layer.
The host completely recovers connectivity if reverted to a Virtual Standard Switch (vSS), regardless of the load balancing policy applied.

Flow of events during ESXi host reboot configured with LAG includes:

NIC Driver Load:
- In boot.log, the line "Ending successfully device probe" confirms that the physical hardware initializes perfectly at Layer 1 before any software policies exist.

/var/run/log/boot:
YYYY-DD-MMT05:39:55.838Z cpu58:2097904) Loading module <driver_module> ...
YYYY-DD-MMT05:39:55.850Z cpu58:2097903) Device: 193: Registered driver '<driver_module>' from 59
YYYY-DD-MMT05:39:55.852Z cpu58:2097903) <driver_module>: <driver_module>_initialize_devname:85: [0000:##:00.0] PCI PF device 16d7:####:####:14e4 detected
YYYY-DD-MMT05:39:55.872Z cpu58:2097903) <driver_module>: <driver_module>_dev_probe_ph2:1868: Ending successfully device probe

LACP Initialization:
- Inside lacp.log, the entry "LACP service is starting..." confirms that the protocol daemon initiates memory allocation while still completely unaware of the upstream switch state.

/var/run/log/lacp.log:
YYYY-DD-MMT05:39:56Z No(29) lacp[2098304]: 4370, LACP service is starting...
YYYY-DD-MMT05:39:56Z No(29) lacp[2098304]: 458, Init uplink vmnic# with addr <REDACTED_MAC>

The Fail-Open Window:
- The critical vSwitch entries showing "FRP is not yet set on port" followed immediately by "Unblock Port" also confirm that the hypervisor creates a temporary, unmanaged Layer-2 blind spot that lets the initial pings leak through.

/var/run/log/hostd.log:YYYY-DD-MMT05:40:00.119Z cpu15:2098615) vswitch: L2Sec_EnforcePortCompliance:181: FRP is not yet set on port 4####06
YYYY-DD-MMT05:40:00.121Z cpu26:2097747) vswitch: VSwitchPortEthFRPUpdateInt:5725: Unblock Port 10#####03
YYYY-DD-MMT05:40:01.009Z cpu26:2097747) netschedHClk: NetSchedHClkNotify:5061: vmnic#: link up notification
YYYY-DD-MMT05:40:01.009Z cpu26:2097747) Uplink: 13037: enabled port 0x84000011 with mac <REDACTED_MAC> [vmnic#]

The LACP group (LAG) Collapse:
- After the userworld (hostd) fully initializes, the LACP daemon detects that the physical switch is mis-configured (since Static Mode ON ignores LACPDUs). To avoid network loops, it indefinitely suspends the physical ports (/var/log/lacp.log):
- The final entries showing "Starting service lacp" followed immediately by "Receive uplink... event (port blocked)" confirm that the moment hostd completes its load, the active LACP daemon wakes up, identifies the peer mismatch, and drops the hammer to prevent a loop.

/var/run/log/vmkernel.log: YYYY-MM-DDT05:40:59.068Z In(182) vmkernel: cpu29:2100966)Starting service lacp
YYYY-MM-DDT05:40:59.081Z In(182) vmkernel: cpu29:2100966)Jumpstart plugin lacp activated.

/var/run/log/lacp.log:
YYYY-DD-MMT05:41:05Z No(29) lacp: 3602, Receive uplink vmnic# event (port blocked), smVars 0x3
YYYY-DD-MMT05:41:05Z No(29) lacp: 2820, Remove uplink vmnic# from lag DR lag1(25######06) of portset DvsPortset-0
YYYY-DD-MMT05:41:05Z Er(27) lacp: 2329, Failed to send lacpdu to uplink vmnic#

Environment

VMware vSphere ESXi
VMware vCenter Server

Cause

The primary issue is because of the configuration mismatch between the vDS (configured for dynamic LACP) and the upstream physical switch (configured for Static EtherChannel / Mode ON).

The confusing transient ICMP reachability experienced during ESXi reboot is an expected early-boot behavior. During early VMkernel boot, the hardware network drivers initialize the physical NICs before the management control plane (hostd) has fully loaded the vDS Filter Rules Policy (FRP) from the local state database (/etc/vmware/dvsdata.db or via the Configuration Store repository depending on vSphere ESXi version). Because these strict security and aggregation rules are not yet enforced, the virtual switch engine executes a fail-safe initialization exception (FRP is not yet set on port), temporarily unblocking the port and allowing unmanaged, raw traffic (like ICMP) to bypass restrictions. The duration of this ping recovery is completely reliant on how long hostd takes to load into memory, not LACP slow/fast timers.

Once the hostd jumpstart framework finishes loading the userworld state, the LACP daemon reinitializes and attempts to negotiate with the upstream switch. Because the physical switch is hard-coded to Static Mode ON, it does not respond to LACP Data Units (LACPDUs). The ESXi host detects this protocol failure, strictly enforces loop-prevention mechanics, and suspends the ports, resulting in sudden and total network isolation.

Note: Migrating to a vSS bypasses the LACP daemon and strict dynamic negotiation rules entirely, which is why a vSS temporarily "resolves" the connectivity issue despite the upstream physical switch misconfiguration.

Resolution

To resolve the permanent disconnection following the initialization phase, align the port binding and load balancing configurations identically across the VMware vDS and the upstream physical switch cluster:

If using Dynamic LACP:

Ensure the vDS LAG is configured for LACP (Active/Passive) and the physical switch port-channel is configured for dynamic LACP negotiation (e.g., mode active).

If using Static EtherChannel:

Remove the physical adapters from the vDS LACP LAG. Assign them as standard vDS uplinks, configure the portgroup Teaming and Failover policy to Route based on IP Hash, and ensure the physical switch is configured to Static EtherChannel (mode on).

Emergency Out-of-Band Host Recovery Procedure, (post reboot):

To recover the degraded ESXi host, execute the below steps to move the networking from the distributed to the standard switch:
- Access the ESXi host Shell CLI via out-of-band virtual console (like iDRAC, KVM, iLO).
- Temporarily isolate the management vmkernel interface (vmk#) from the vSphere Distributed Switch (vDS) and attach it to a vSphere Standard Switch (vSS):

esxcfg-vswitch -l <-- Note the port number for the vmnics configured
esxcfg-vswitch -Q vmnic# -V <###> <vDS_Name> <-- Replace it with associated port number from above command output
esxcfg-vswitch -Q vmnic# -V <###> <vDS_Name> <-- Replace it with associated port number from above command output
esxcli network ip interface remove -i vmk# <-- Update vmk# with the management vmkernel configured on the host
esxcli network vswitch standard add -v <standard_switch_name>
esxcli network vswitch standard portgroup add -p <management_portgroup_name> -v <standard_switch_name>
esxcli network vswitch standard portgroup set -p <management_portgroup_name> -v <portgroup_VLAN>
esxcli network vswitch standard uplink add -u vmnic# -v <standard_switch_name>
esxcli network vswitch standard uplink add -u vmnic# -v <standard_switch_name>
esxcli network vswitch standard portgroup policy failover set -l iphash -p <management_portgroup_name>
esxcli network vswitch standard portgroup policy failover set -a vmnic# -a vmnic# -p <management_portgroup_name>
esxcli network ip interface add -i vmk# -p <management_portgroup_name>
esxcli network ip interface ipv4 set -i vmk# -I <###.##.##.#> -N <###.##.##.#> -g <###.##.##.#> -t static <-- IP address of the host of vmk# that is going through the migration/upgrade.
esxcli network ip interface tag add -i vmk# --tagname=management
esxcfg-route -a default <###.##.##.#>

Additional Information

To speak with a customer representative or a Support Engineer see Contact Support: Contact Support

For more information, refer the articles: