FDM Manager restarts on an ESXi host after vMotion of a VM has completed

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• In the vCenter Server the following High Availability (HA) alarm event is notified:

vSphere HA agent for this host has an error.  The vSphere HA agent is not reachable from vCenter server.
​​​​​

• Within the vmkernel.log of the ESXi host you see an fdm-zdump.000 is generated to /var/core/

2019-01-01T11:06:26.342Z cpu11:71565)UserDump: 3024: worker: Dumping cartel 71561 (from world 73525) to file /var/core/fdm-zdump.000
 

• Within the the fdm.log on the ESXi host you see the following pattern of logs just before FDM restarts.

2019-01-01T11:06:26.325Z verbose fdm[6481B70] [Originator@6876 sub=Invt] [HalVmMonitor::HandleVmGuestKernelCrashedChange] GuestKernelCrashed is false for VM 132
2019-01-01T11:06:26.325Z verbose fdm[6481B70] [Originator@6876 sub=Invt] [HalVmMonitor::Notify] VM 132: Updated GuestKernelCrashed!
2019-01-01T11:06:26.326Z verbose fdm[6C81B70] [Originator@6876 sub=Invt opID=SWI-611ab062] [VmStateChange::SaveToInventory] vm /vmfs/volumes/770c5945-e013a728-c066-20a8e0220d66/VMname/VMname.vmx from __localhost__ changed inventory  datastoresInUse changed overhead=0 dvsSwitchMap= vnicResvDec=0 vnicResInfo= numCpu=1
2019-01-01T11:06:26.327Z verbose fdm[6FDBB70] [Originator@6876 sub=Invt] [HalVmMonitor::HandleVmRemoval] Removing vm 132
2019-01-01T11:06:26.328Z info fdm[7640B70] [Originator@6876 sub=Invt opID=SWI-14c81ba4] [InventoryManagerImpl::RemoveVmLocked] vm /vmfs/volumes/770c5945-e013a728-c066-20a8e0220d66/VMname/VMname.vmx (not protected) removed from local host; on 0 hosts
2019-01-01T11:06:26.327Z verbose fdm[61D1B70] [Originator@6876 sub=Policy opID=SWI-1f3223bc] [LocalVmcpPolicy::RemoveVmConfig] Removed VM /vmfs/volumes/770c5945-e013a728-c066-20a8e0220d66/VMname/VMname.vmx
2019-01-01T11:06:26.329Z verbose fdm[7640B70] [Originator@6876 sub=Invt opID=SWI-14c81ba4] [InventoryManagerImpl::RemoveVmLocked] Unprotected vm /vmfs/volumes/770c5945-e013a728-c066-20a8e0220d66/VMname/VMname.vmx gone from the cluster
2019-01-01T11:06:26.330Z verbose fdm[7640B70] [Originator@6876 sub=Invt opID=SWI-14c81ba4] [VmStateChange::SaveToInventory] vm /vmfs/volumes/770c5945-e013a728-c066-20a8e0220d66/VMname/VMname.vmx removed from __localhost__
2019-01-01T11:07:10.537Z Section for VMware Fault Domain Manager, pid=5274999, version=6.5.0, build=8024368, option=Release
2019-01-01T11:07:10.537Z verbose fdm[50E4940] [Originator@6876 sub=Default] Dumping early logs:

Follow this article for: vSphere HA reports that an agent is in the Agent Unreachable state

Environment

VMware vSphere ESXi 6.5
VMware vCenter Server 6.5.x

Cause

This issue occurs due to a rare race condition within FDM during a vMotion event.

Resolution

This issue is resolved in VMware vCenter Server 6.5 Update 3, available at VMware Downloads .