ESXi Host Disconnection During Rubrik Backups Over 1 Gbps Management Uplinks

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms

• ESXi hosts intermittently show as "Not Responding" or disconnect from vCenter during the Rubrik backup window.
• The Rubrik backup appliance mounts transient NFS datastores that are not pinned to a specific high-speed VMkernel port.
• In hostd.log, you observe NFC_NETWORK_ERROR and "Broken pipe" messages.
• Standard VMkernel port binding for NFS storage is ineffective because the datastore is temporary (mounted/unmounted per job).

Environment

ESXi (All versions) 

Cause

Heavy NFS backup traffic is being dynamically routed over the 1 Gbps Management interface instead of the intended high-speed (e.g., 10 Gbps/25 Gbps) storage uplinks. This saturates the management path, causing heartbeats to fail between the host and vCenter, leading to host isolation or disconnection.

Resolution

To ensure backup traffic is segregated from management traffic when using transient NFS mounts, implement one of the following:

Option 1: Physical Uplink Remediation (Preferred)

Remove the 1 Gbps interfaces from the active uplink team for the VDS/Standard Switch. If management and storage must share a switch, ensure all active uplinks are of uniform high speed to prevent the 1 Gbps link from becoming a bottleneck during path selection.

Option 2: Logical Segregation (Routing/VLAN Tagging)

1. Isolate Networks: Place the Rubrik backup interfaces and the ESXi Storage VMkernel ports on a dedicated VLAN/Subnet separate from the Management network.
2. Define Static Routes: If the networks must communicate across Layer 3, configure static routes on the ESXi host to force traffic destined for the Rubrik IPs to exit via the high-speed Storage VMkernel adapter rather than the default gateway on the Management VMkernel.
3. VDS Port Group Overrides: Configure the VDS Port Group for the backup network to explicitly use only the high-speed fiber uplinks in its "Teaming and Failover" policy, marking the 1 Gbps links as "Unused."

 

Note on VMkernel Port Binding (vSphere 8.0 U1+): If the NFS datastore is not transient (i.e., it remains persistently mounted), and you are running ESXi 8.0 Update 1 or later, you can utilize the native VMkernel port binding feature to force NFS traffic onto specific adapters.

• For NFS v3, port binding is available starting in 8.0 Update 1 ESXCli command to update VMKernel binding of an NFSv3 datastore.
• For NFS v4.1, port binding (vmkportbind) is available starting in 8.0 Update 3 NFS 4.1 Datastores using Custom NFS TCP/IP Stack become inaccessible after upgrading to ESXi 8.0.2.

Refer to NFS Support in VMware vSphere 8.0 and Beyond for detailed configuration steps and performance enhancements like nConnect.