VMware vSphere 5.x High Availability (FDM) fails to configure and a timeout is reported during HA election

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• The vSphere HA availability state of this host has changed to Election
   
• Enabling VMware HA on vCenter Server 5.0 fails
• HA waits for cluster election to complete and then times out
• In the hostd logs, you see entries similar to:
• In the host log file, you see entries similar to:
   
• • YYYY-MM-DDT00:29:22.775Z [25840B90 info 'TaskManager' opID=F2AD2ACE-00000C1D-95-a6] Task Created : haTask-ha-host-vim.host.ServiceSystem.updatePolicy-281
    YYYY-MM-DDT00:29:22.775Z [25E9CB90 verbose 'ServiceSystem' opID=F2AD2ACE-00000C1D-95-a6] Invoking command /sbin/chkconfig vmware-fdm off
    YYYY-MM-DDT00:29:22.776Z [25E9CB90 info 'SysCommandPosix' opID=F2AD2ACE-00000C1D-95-a6] ForkExec(/sbin/chkconfig) 7068
    YYYY-MM-DDT00:29:22.802Z [25E9CB90 verbose 'ServiceSystem' opID=F2AD2ACE-00000C1D-95-a6] Command finished with status 0
    YYYY-MM-DDT00:29:22.802Z [25E9CB90 error 'Default' opID=F2AD2ACE-00000C1D-95-a6] AdapterServer caught exception: Invalid argument : Cannot create path vmware-fdm. Element service already has value
    YYYY-MM-DDT00:29:22.802Z [25E9CB90 error 'Default' opID=F2AD2ACE-00000C1D-95-a6] Backtrace
    Enabling VMware HA on vCenter Server 5.0 fails
     
  • YYYY-MM-DDT00:29:22.803Z [25E9CB90 info 'TaskManager' opID=F2AD2ACE-00000C1D-95-a6] Task Completed : haTask-ha-host-vim.host.ServiceSystem.updatePolicy-281 Status error
    YYYY-MM-DDT00:29:22.803Z [25E9CB90 info 'Vmomi' opID=F2AD2ACE-00000C1D-95-a6] Activation [N5Vmomi10ActivationE:0x4f7bd88] : Invoke done [updatePolicy] on [vim.host.ServiceSystem:serviceSystem]
    YYYY-MM-DDT00:29:22.803Z [25E9CB90 verbose 'Vmomi' opID=F2AD2ACE-00000C1D-95-a6] Arg id:
    --> "vmware-fdm"
    YYYY-MM-DDT00:29:22.803Z [25E9CB90 verbose 'Vmomi' opID=F2AD2ACE-00000C1D-95-a6] Arg policy:
    --> "off"
    YYYY-MM-DDT00:29:22.803Z [25E9CB90 info 'Vmomi' opID=F2AD2ACE-00000C1D-95-a6] Throw vmodl.fault.SystemError
    YYYY-MM-DDT00:29:22.803Z [25E9CB90 info 'Vmomi' opID=F2AD2ACE-00000C1D-95-a6] Result:
    --> (vmodl.fault.SystemError) {
    --> dynamicType = <unset>,
    --> faultCause = (vmodl.MethodFault) null,
    --> reason = "",
    --> msg = "",
    --> }

Environment

VMware vCenter Server 5.0.x
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.0

Resolution

This is a known issue affecting VMware vCenter Server 5.0.

 

To workaround this issue, manually uninstall the FDM agent.

 

To manually uninstall the FDM agent:

1. Place the ESXi host in Maintenance Mode.
2. Run this command to copy /opt/vmware/uninstallers/VMware-fdm-uninstall.sh to the /tmp directory:
   
   # cp /opt/vmware/uninstallers/VMware-fdm-uninstall.sh /tmp
    
3. Run this command to assign execute permission to the VMware-fdm-uninstall.sh file.
   
   # chmod +x /tmp/VMware-fdm-uninstall.sh
    
4. Run this command to uninstall the FDM agent:
   
   # /tmp/VMware-fdm-uninstall.sh
   
   Note: You should see a short pause before you see the prompt again.
    
5. Exit the host from Maintenance Mode.
   
   In Recent Tasks, you should see that the HA agent is pushed from vCenter Server and installed.

Additional Information

Cannot install Fault Domain Manager agent for VMware HA after agent is uninstalled

Impact/Risks:
The VMware-fdm-uninstall.sh uninstall script does not remove the package properly when it is run from the /opt/vmware/uninstallers/ directory. When HA is disabled from vCenter Server, the script is copied to /tmp and is run from there. When manually uninstalling the FDM agent, ensure that it is copied to and executed from the /tmp directory.