HCX Bulk Migration Stuck at 0% Base Sync Due to Network Profile MTU Mismatch
Article ID: 386191
Updated On:
Products
Issue/Introduction
HCX Bulk Migration can get stuck at 0% during base sync; if it does run the diagnostic test on the service mesh. It is possible there is a MTU mismatch in the configuration causing diagnostic probes to fail, showing the following errors
Error: Get "https://ip-address/sdk/vimService.wsdl": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Message: HCX-WAN-IX is unable to reach Service_HostSystem on the ports 80. Please ensure firewall is not blocking the ports or routing is correctly configured.
This error indicated that the HCX-WAN-IX was unable to connect to the ESXi hosts on port 80, suggesting a network issue. The other part of the error which is exceeded while awaiting headers implies that while waiting to read a header the client timed out due to context deadline exceeded which points towards MTU.
Probe Failure
Environment
- VMware HCX
Cause
This issue occurs due to network misconfigurations, specifically related to MTU (Maximum Transmission Unit) settings in the network profile. The HCX-WAN-IX is unable to connect to the ESXi hosts on port 80, as indicated by the diagnostic error "context deadline exceeded" and failure to reach the Service_HostSystem. This error pointed to timeouts during communication, often caused by packet fragmentation, which can happen when the MTU settings are inconsistent.
Note: This is generally seen when the HCX network profile is configured for jumbo frames and the VMK on at least one of the ESXi hosts in the connection or a switch is configured for standard frames and not jumbo frames causing fragmentation.
Resolution
To resolve the issue, verify and adjust the MTU setting in the network profile to match the MTU configuration of the ESXi hosts and the underlying network infrastructure. Ensuring that the MTU value is consistent across the network profile, the ESXi hosts, and any relevant network devices will eliminate packet fragmentation. After making these adjustments, the diagnostic probes should succeed, and the HCX Bulk Migration will progress beyond the 0% Base Sync phase. If they do not please reach out to Broadcom Support and note this article.
Additional Information
Feedback
