Understanding IP Hash load balancing

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides the advantages and disadvantages of using IP Hash NIC load balancing compared to Originating Port ID load balancing.

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Resolution

The NIC teaming section plays a vital role when you have two or more uplinks (VMNICs). To determine the current load balancing configuration on a vSphere Standard Switch:

• Using the vSphere Client, navigate to Host and Clusters > Host Name > Configuration tab.
• Click Virtual switches under Networking.
• Click the vSwitch, click Edit and navigate to the "Teaming and failover" for load balancing configuration.

 

Route Based on Originating Port ID

This is the load balancing method that is used by default on vSphere Standard and Distributed Switches. When this policy is in effect, only one VMNIC (physical NIC) is used per virtual NIC or VMkernel port. If a NIC's link goes down, the virtual NICs and VMkernel ports are reassigned to the remaining NICs that are up, and the host sends out advertisements to the physical switch to ensure the MAC address table is updated.

If all NICs in a team are set as active, they are all used for virtual machine or VMkernel traffic.

Advantages:

• Simple physical switch configuration. No link bonding (EtherChannel) is required. Only independent switch ports require configuration.
• Network adapter redundancy. Even when all NICs are in active use, if a NIC fails, the remaining NICs in the team continue passing traffic.
• Links in a team can be spread across multiple physical switches for hardware failure redundancy.
• Link Status and Beacon Probing failover detection methods can be used with this type of load balancing.
• In an environment with multiple virtual machines, load is spread across all active NICs for better performance.

Disadvantages:

• A single virtual NIC cannot use more than a single physical NIC's bandwidth. For example, if four 1 Gb NICs are in a team, a virtual machine with a single virtual NIC cannot use more than 1 Gb of bandwidth via a single adapter.

 

Route Based on IP Hash

Route based on IP Hash works by taking the source and destination IP addresses and performing a mathematical calculation on each packet to determine which uplink in the team to use. Because the load balancing is based on the source/destination IP addresses, a single virtual machine communicating with multiple IP addresses can balance its load across all of the network adapters in the team and make better use of the available bandwidth.

Route based on IP Hash load balancing requires that the physical switch ports be combined into an EtherChannel (sometimes called an aggregation bond, port-channel). This ensures that the same hashing algorithm is used for traffic returning in the opposite direction.

Advantages:

• Improved performance in situations where a single virtual machine communicates with multiple virtual machines. Theoretically, a virtual machine can use more bandwidth than a single physical NIC can provide.
• Network adapter redundancy: If a NIC fails and the link-state goes down, the remaining NICs in the team continue to pass traffic. For the link-state to work properly, both the physical switch and the ESXi host must recognize the link as down. If there is a discrepancy, traffic may not fail-over to other adapters in the team.

Disadvantages:

• Physical switch configuration is less flexible and requires that physical switch ports be configured for a static EtherChannel bond. For more information, see Enable EtherChannel / Link Aggregation Control Protocol (LACP) in ESXi/vCenter.
• Only a single physical switch can be used for a NIC team because most switches do not support EtherChannel bonds across multiple physical switches. This prevents physical hardware redundancy.
  
  Note: There are some exceptions, as some "stacked" or modular switches can do this across multiple physical switches or modules. Cisco's VPC (virtual port channel) technology can also address this on supported switches. Contact your hardware vendor for more information.
   
• Beacon probing is not supported with IP Hash. Only link status can be used as a failure detection method. If a link fails without the link state going down, there is no way to avoid network communication issues on the vSwitch.
• Only Static mode EtherChannel is supported with Standard vSwitches. LACP is supported with vSphere Distributed Switches.

Additional Information

For more information on Load Balancing Policies available for virtual switches following article can be referred:
Load Balancing Algorithms Available for Virtual Switches

If you encounter a situation where the virtual MACs are being learnt on both switch interfaces randomly causing interfaces to flap at switch level, this is a sign that a host in your environment is not properly configured. You will need to re-validate the TOR EtherChannel configuration and make sure that it matches the ESXI VDS config and the correct algorithm is selected.