VMs Lose Network Connectivity After vMotion When Service Insertion VM Hangs
Article ID: 433475
Updated On:
Products
Issue/Introduction
- Guest virtual machines intermittently lose Layer 3 network connectivity (such as ICMP ping drops) while Layer 2 (ARP) remains functional.
- This behavior is typically observed after a vMotion event or shortly after a VM boots on an overlay network.
- Symptoms include VMs connecting briefly before traffic is silently dropped, resulting in the appearance that the VMs are "falling off the network."
Environment
Third-Party Service Insertion (e.g., Gigamon, Trend Micro)
Cause
The virtual Service Insertion Platform (vSIP) kernel module on the ESXi host redirects traffic to a local Service VM (SVM) using the vmware-si IOChain filter.
If the target SVM becomes hung or unresponsive but remains powered on, the vSIP module registers the vmState as Attached.
Consequently, the ESXi host continues to punt packets to the unresponsive SVM, creating a traffic blackhole.
The configured failurePolicy (such as failOpen) is not triggered because the host still detects the SVM as actively attached.
Resolution
-
Identify the ESXi host where the impacted guest virtual machine is currently running.
-
Establish an SSH session to the ESXi host.
-
Validate the IOChain filter state for the impacted VM's virtual NIC by executing the following command:
-
summarize-dvfilter | grep -i "vmware-si" -A 10
-
-
Review the output to confirm if traffic is being punted to a Service VM (Slot 12).
-
Identify the associated third-party SVM on the host and check its console for hung states or boot failures.
-
To immediately bypass the blackhole and restore network connectivity, power off the hung SVM. This forces the ESXi host to register the
vmStateasDetached, triggering thefailOpenpolicy to allow traffic to pass natively. -
Engage the third-party vendor to investigate the root cause of the Service VM instability.
Additional Information
Feedback
