"Bad Parameter" error on Fault Tolerance with multiple NIC configuration

Products

• VMware vSphere ESXi

Issue/Introduction

• An ARP Flux issue may occur when the Fault Tolerance network is configured with multiple NICs in the same vmkernel subnet. As a result, a socket "Bad Parameter" error may occur in the vmkernel, which may break the connection between the Primary VM and Secondary VM, and unnecessary Fault Tolerance failover occurs.
• In the /var/log/vmerkernel.log file of the ESXi host, you see entries similar to:
  
  2020-08-28T05:11:52.518Z cpu56:144353014)WARNING: FTCpt: 5230: (25749962114582 pri) Error writing chunk header: Bad parameter
  2020-08-28T05:11:52.518Z cpu56:144353014)WARNING: FTCpt: 5300: (25749962114582 pri) Error transmitting pages (15140 bytes, 1 ms elapsed): Bad parameter
  2020-08-28T05:11:52.518Z cpu49:144353015)WARNING: FTCpt: 4079: (25749962114582 pri) Error on tx queue 1: Bad parameter
  2020-08-28T05:11:52.518Z cpu49:144353015)WARNING: FTCpt: 8505: (25749962114582 pri) Error waiting for sending checkpoint data: Bad parameter
  2020-08-28T05:11:52.518Z cpu49:144353015)WARNING: FTCpt: 8506: (25749962114582 pri) Remote not responding
  
  Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.5
VMware vSphere ESXi 7.0.0
VMware vSphere ESXi 6.7

Resolution

To resolve this issue, use different subnets for each vmkernel interface when implementing a multiple NIC configuration in a Fault Tolerance network.

For example:

Correct configuration of subnets 

FT_vmkernel1:192.168.10.10 / 255.255.255.0
FT_vmkernel2:192.168.11.10 / 255.255.255.0
FT_vmkernel3:192.168.12.10 / 255.255.255.0
FT_vmkernel4:192.168.13.10 / 255.255.255.0

Incorrect configuration of subnets

FT_vmkernel1:192.168.10.10 / 255.255.255.0
FT_vmkernel2:192.168.10.11 / 255.255.255.0
FT_vmkernel3:192.168.10.12 / 255.255.255.0
FT_vmkernel4:192.168.10.13 / 255.255.255.0

Additional Information

• Multi-homing on ESXi/ESX (2010877)
• Best Practices for Fault Tolerance