ESX and ESXi hosts in a HA cluster randomly display the message: A possible host failure has been detected on in cluster in datacenter

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• In the Summary tab of the ESX/ESXi host in a HA cluster, you see this message:
  
  A possible host failure has been detected on <host> in cluster <cluster> in <datacenter> datacenter
  
  
• Reconfiguring HA on the affected host temporarily corrects this issue.
• In the /var/log/vmware/aam/vmware_<hostname>.log file of one or more ESX/ESXi hosts, you see entry similar to:
  
  MESSAGE: Message received from an unexpected sender: msgid 621 from (1/0:rtcp_1.0)
  
  
• In the /var/log/vmkernel log file of the ESX host (/var/log/messages log file of the ESXi host), you see entry similar to:
  
  Tcpip_Vmk: 143: arp: xx:xx:xx:xx:xx:xx is using my IP address 192.168.1.50!
  warning 'HostsvcPlugin'] Duplicate IP 192.168.1.50 detected at mac xx:xx:xx:xx:xx:xx
  
  
• In the /var/run/log/vobd.log file of the ESXi 5.5 host, you see entry similar to:
  
  2017-01-18T18:25:13.581Z: [netCorrelator] 5716995306931us: [vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for 10.xx.xx.xx on interface vmk0. The current owner is f0:1f:af:d8:ad:55.
  2017-01-18T18:25:13.581Z: [netCorrelator] 5717119985527us: [esx.problem.net.vmknic.ip.duplicate] Duplicate IP address detected for 10.xx.xx.xx on interface vmk0, current owner being f0:1f:af:d8:ad:55
  
  
  

Environment

VMware vCenter Server 5.0.x
VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Embedded
VMware ESXi 4.1.x Installable
VMware ESXi 3.5.x Installable
VMware vSphere ESXi 5.0
VMware vCenter Server 4.1.x
VMware ESX 4.1.x
VMware ESX Server 3.5.x
VMware ESX Server 3.0.x
VMware ESXi 4.0.x Installable
VMware ESX 4.0.x
VMware ESXi 3.5.x Embedded
VMware vCenter Server 4.0.x

Cause

This issue occurs when a computer on the network is configured with the same IP address as one of the ESX or ESXi hosts in a HA cluster. In such a case:

• the HA agent receives invalid data and generates errors
• the HA agent does not function properly until it is reconfigured
• the frequency of this issue depends on the frequency of the IP conflict encountered

Resolution

To resolve this issue, find the computer with the duplicate IP address and configure it with a different IP address.

You can use the Duplicate IP warning in the /var/log/vmkernel log file of the ESX host, or the /var/log/messages log file of the ESXi host to identify the computer with the duplicate IP address.

Note: Duplicate IP may or may not be available in the messages or vmkernel.log. In such a case, reconfigure HA at the cluster or host level.

Additional Information

To be alerted when this document is updated, click the Subscribe to Article link in the Actions box