vCenter HA failover happened with Failed to read health xml file error on health check on vcha

Products

• VMware vCenter Server

Issue/Introduction

vCenter HA failover happened without reason.

From the vmon log in previous active vcenter, you can see below log entry

Error: Failed to read health xml file: /etc/vmware-vcha/vcha_health.xml. Error: time data '[DATE]T23:54:35Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

- /var/log/vmware/vmon.log

[DATE/TIME] In(05) host-16370 <vcha-healthcmd> Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonApiHealthCmd.py -n vmware-vcha -f /etc/vmware-vcha/vcha_health.xml
[DATE/TIME] Wa(03) host-16370 <vcha> Service api-health command's stderr: Error getting service health. Error: Failed to read health xml file: /etc/vmware-vcha/vcha_health.xml. Error: time data '[DATE]T23:54:35Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'
[DATE/TIME] Wa(03)+ host-16370
[DATE/TIME] Wa(03) host-16370 <vcha> Service api healthcheck command returned unknown exit code 1
[DATE/TIME] Er(02) host-16370 <vcha> health state unknown ,considered as system failure
[DATE/TIME] Er(02) host-16370 System Failure, initiating system restart.

Environment

vCenter Server 8.0.3 

Cause

As you can see below log, the time value does not have msec and usec value.

Error: Failed to read health xml file: /etc/vmware-vcha/vcha_health.xml. Error: time data '[DATE]T23:54:35Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

 

It is violation for parsing date and the possibility of this issue can be only one value from our code side.

The value is like '[DATE]T23:54:35.000.000Z'

Resolution

There is no workaround and this can happen at a very rare probability.

The fix will be in future release.