"Bad Parameter" error on Fault Tolerance with multiple NIC configuration

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• An ARP Flux issue may occur when the Fault Tolerance network is configured with multiple NICs in the same vmkernel subnet. As a result, a socket "Bad Parameter" error may occur in the vmkernel, which may break the connection between the Primary VM and Secondary VM, and unnecessary Fault Tolerance failover occurs.
• In the /var/log/vmerkernel.log file of the ESXi host, you see entries similar to:

YYYY-MM-DDTHH:MM:SS.518Z cpu56:144353014)WARNING: FTCpt: 5230: (25749962114582 pri) Error writing chunk header: Bad parameter
YYYY-MM-DDTHH:MM:SS.518Z cpu56:144353014)WARNING: FTCpt: 5300: (25749962114582 pri) Error transmitting pages (15140 bytes, 1 ms elapsed): Bad parameter
YYYY-MM-DDTHH:MM:SS.518Z cpu49:144353015)WARNING: FTCpt: 4079: (25749962114582 pri) Error on tx queue 1: Bad parameter
YYYY-MM-DDTHH:MM:SS.518Z cpu49:144353015)WARNING: FTCpt: 8505: (25749962114582 pri) Error waiting for sending checkpoint data: Bad parameter
YYYY-MM-DDTHH:MM:SS.518Z cpu49:144353015)WARNING: FTCpt: 8506: (25749962114582 pri) Remote not responding

• Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 
VMware vCenter Server

Resolution

To resolve this issue, use different subnets for each vmkernel interface when implementing a multiple NIC configuration in a Fault Tolerance network.

Example configuration:

Correct configuration of subnets 

FT_vmkernel1: 192.168.10.10 / 255.255.255.0
FT_vmkernel2: 192.168.11.10 / 255.255.255.0
FT_vmkernel3: 192.168.12.10 / 255.255.255.0
FT_vmkernel4: 192.168.13.10 / 255.255.255.0

Incorrect configuration of subnets

FT_vmkernel1: 192.168.10.10 / 255.255.255.0
FT_vmkernel2: 192.168.10.11 / 255.255.255.0
FT_vmkernel3: 192.168.10.12 / 255.255.255.0
FT_vmkernel4: 192.168.10.13 / 255.255.255.0

Additional Information

• For more reference on best practices for Fault Tolerance, please refer: Best Practices for Fault Tolerance
• For more information regarding multi-homing, please refer: Multihoming on ESXi