vCenter High Availability failover from the passive node back to the active node leaves VCHA inoperable.

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• vCenter HA is able to failover the passive node, but failing back over to the primary leaves the passive offline.
• A snapshot was taken of the vCenter Appliance while VCHA was enabled.
• During the failed failover, vMon restarts the passive node without the "HACore" profile in /var/log/vmware/vmon/vmon-syslog.log

    notice vmon  Starting vMon with profile 'NONE'

• There are errors refering to the prefreeze script being unable to create a partnerlist file in /var/log/preefreeze/prefreeze_#####_#_stderr.log

ERROR:root:Failed to create partnerlist file at the location /storage/db/vmware-vmdir/tmp/.
Stdout Error: Backup requires vmdird service running

Stderr
Traceback (most recent call last):
  File "/usr/lib/vmware/site-packages/create-partner-list.py", line 69, in <module>
    main()
  File "/usr/lib/vmware/site-packages/create-partner-list.py", line 62, in main
    port, domainName = GetDCConfig()
  File "/usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/util/Common.py", line 826, in GetDCConfig
    lsObj = LookupServiceClient(vmafdClient.GetLSLocation(), retry_count=2)
RuntimeError: unidentifiable C++ exception
 

Environment

VMware vCenter Server 6.7.x

Cause

When a snapshot is taken of vCenter, part of the prefreeze script creates a file called backupMarker.txt, but the prefreeze script fails due to VCHA being enabled. During the second failover, the passive node is rebooted. During the boot, the applmgmt service finds the /etc/vmware/backupMarker.txt file and attempts to perform restore. As part of the restore, it masks all services, which prevents the vcha service from coming up.

Resolution

Taking snapshots of a vCenter Server with VCHA enabled is not supported. In some circumstances, this is expected behavior when doing so.

Before attempting a failover operation, ensure that no snapshots exist and remove the /etc/vmware/backupMarker.txt file on any of the vCenter Server HA nodes.

Additional Information

FAQ: vCenter High Availability (2148003)