Multihoming on ESXi

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information about multihoming VMkernel networking.

Multihoming in a VMkernel networking context means that there are multiple VMkernel adapters (vmknic) in a single TCP/IP stack (Netstacks).

Observed issues that may be present:

vMotion operations can fail
Storage latency may be reported (iSCSI not using port binding)
ESXi host disconnects from vCenter can occur.

The above may exist if you have VMkernel ports configured on the same IP subnet and TCP/IP stack similar to:

Managment vmk0 with IP 192.168.0.8/24 and Default Gateway 192.168.0.1 as part of the default TCP/IP stack.
vMotion vmk1 with IP 192.168.0.10/24 and Default Gateway 192.168.0.1 as part of the default TCP/IP stack.
iSCSI vmk2 with IP 192.168.0.12/24 and Default Gateway 192.168.0.1 as part of the default TCP/IP stack.

In the above configuration all of the vmknics are in the same IP subnet and TCP/IP stack. This results in the TCP/IP stack potentially selecting the wrong vmknic for egress vmknic traffic and can lead to potential issues with connectivity, low throughput, and asymmetric routing

Environment

VMware vSphere ESXi

Cause

Configurations with more than one vmknic in the same IP subnet and TCP/IP stack is not supported.

NOTE: there are three exceptions:

iSCSI port binding
Multi-NIC vMotion

See additional information below

Resolution

Ensure the VMkernel ports are in a different IP subnets or have their own TCP/IP stack.

Configurations with more than one vmknic interface on the same IP subnet is not supported. (See addition information for exceptions)

Examples:

Supported VMkernel port configuration:
1. vMotion vmk1 and IP 192.168.0.10/24 with Default Gateway 192.168.0.1/24 (in the pre-defined vMotion TCP/IP stack).
2. iSCSI vmk2 and IP 172.16.0.12/24 with Default Gateway 172.16.0.1/24 (in a Custom TCP/IP stack).
Supported VMkernel port configuration example:
1. Management vmk0 and IP 192.168.0.10/24 with Default Gateway 192.168.0.1/24 (in the pre-defined Default TCP/IP stack).
2. vMotion vmk1 and IP 10.10.128.12/24 with Default Gateway 192.168.0.1/24 (in the pre-defined Default TCP/IP stack).
3. iSCSI vmk2 and IP 172.16.0.12/24 with Default Gateway Override, using gateway 172.16.0.1/24 (in the pre-defined Default TCP/IP stack).

Additional Information

For more information about TCP/IP stacks in vSphere ESXi, see View TCP/IP Stack Configuration on a Host
For more information and exceptions about iSCSI port binding, see Considerations for using software iSCSI port binding in ESX/ESXi
For more information and exceptions about Multiple-NIC vMotion, see Multiple-NIC vMotion in vSphere
For more information about VMkernel Networking in vSphere, see VMkernel Networking Layer

NOTE: Configurations with more than one vmknic interface on the same IP subnet is not supported.