ESXi hosts in a HA cluster randomly display the message: A possible host failure has been detected on in cluster in datacenter

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• In the Summary tab of the ESXi host in a HA cluster, you see this message:
  
  A possible host failure has been detected on <host> in cluster <cluster> in <datacenter> datacenter
  
  
• Reconfiguring HA on the affected host temporarily corrects this issue.
• In the /var/log/vmware/aam/vmware_<hostname>.log file of one or moreESXi hosts, you see entry similar to:
  
  MESSAGE: Message received from an unexpected sender: msgid 621 from (1/0:rtcp_1.0)
  
• In the /var/log/vmkernel.log log file of the ESXi host, you see entry similar to:
  
  Tcpip_Vmk: 143: arp: xx:xx:xx:xx:xx:xx is using my IP address 192.168.1.50!
  warning 'HostsvcPlugin'] Duplicate IP 192.168.1.50 detected at mac xx:xx:xx:xx:xx:xx
  
  
• In the /var/run/log/vobd.log file of the ESXi host, you see entry similar to:
  
  2017-01-18T18:25:13.581Z: [netCorrelator] 5716995306931us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for 10.xx.xx.xx on interface vmk0. The current owner is f0:1f:af:d8:ad:55.
  2017-01-18T18:25:13.581Z: [netCorrelator] 5717119985527us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 10.xx.xx.xx on interface vmk0, current owner being f0:1f:af:d8:ad:55
  
  
  

Environment

VMware vSphere ESXi 6.*
VMware vSphere ESXi 7.0
VMware vSphere ESXi 8.0
VMware vSphere vCenter 6.*
VMware vSphere vCenter 7.0
VMware vSphere vCenter 8.0

Cause

This issue occurs when a computer on the network is configured with the same IP address as one of the ESXi hosts in a HA cluster. In such a case:

• The HA agent receives invalid data and generates errors
• The HA agent does not function properly until it is reconfigured
• The frequency of this issue depends on the IP conflict encountered

Resolution

To resolve this issue, find the computer with the duplicate IP address and configure it with a different IP address.

You can find the Duplicate IP warning in the /var/log/vmkernel.log log file of the ESXi host to identify the computer with the duplicate IP address.

Note: Duplicate IP may or may not be available in the messages or vmkernel.log. In such a case, reconfigure HA at the cluster or host level.