With two hosts that are directly connected to each other (not connected to a switch), a scenario might occur where the driver of the vmnic on host "A" reports the linkstate down to ESXi on that host, but the actual/physical linkstate remains up.
This inconsistency results in host "B" seeing the link as up, which prevents failover to another vmnic, as ESXi does not have a mechanism to make one host "aware" of the "logical down" linkstate on another host.
The linkstate is controlled by the driver, not ESXi.
Contact your hardware vendor for assistance.
See: Network Failure Detection Policy and Link state up or down for a vmnic interface on ESXi for additional details.