Multiple VMkernel interfaces on the same subnet must have the same configuration for MTU/VLAN to avoid issues. This issue occurs if any one of the VMkernel interface configuration is not identical.
For example, when VMKernel interface vmk1 used for Storage Hearbeat is not configured with the correct MTU, running esxcfg-vmknic -l command, you see output similar to:
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk0 Management Network IPv4 10.###.###.# 255.255.255.0 10.###.###.255 ##:##;##;##;##;## 1500 65535 true STATIC
vmk1 Storage Heartbeat IPv4 10.###.###.# 255.255.255.0 10.###.###.255 ##:##;##;##;##;## 1500 65535 true STATIC
vmk2 iSCSI1 IPv4 10.###.###.# 255.255.255.0 10.###.###.255 ##:##;##;##;##;## 9000 65535 true STATIC
vmk3 iSCSI2 IPv4 10.###.###.# 255.255.255.0 10.###.###255 ##:##;##;##;##;## 9000 65535 true STATIC