Unexpected VCHA failover could happen due to health check failure of vc-ws1a-broker.

Products

• VMware vCenter Server

Issue/Introduction

VCHA failover happens as an active node restarts due to health check failure of vc-ws1a-broker and it is seen in /var/log/vmware/vmon/vmon.log.

<DATE_TIME> In(05) host-2503 <vc-ws1a-broker> Running the API Health command as user <DATE_TIME> In(05) host-2503 <vc-ws1a-broker-healthcmd> Constructed command: /storage/containers/vc-ws1a-broker/dbb7fed8d494caf9bf2ee657a3622d62b846347019b4e13df5f0bd59128010c0/health_check.sh <DATE_TIME> Wa(03) host-2503 <vc-ws1a-broker> Service api healthcheck command returned unknown exit code 63 <DATE_TIME> Er(02) host-2503 <vc-ws1a-broker> health state unknown ,considered as system failure <DATE_TIME> Er(02) host-2503 System Failure, initiating system restart.

Environment

VMware vCenter Server

Cause

It is caused that reading health check file fails and it is seen in /var/log/vmware/vc-ws1a-broker/nginx-error.log.

<DATE_TIME> [alert] 78#0: *53991 pread() read only 0 of 5 from "/opt/vmware/idm/health/healthCheck" while sending response to client, client: ::ffff:127.0.0.1, server: localhost, request: "GET /sddc/broker/health HTTP/1.1", host: "localhost:10121"

Resolution

This issue is fixed in vCenter server 8.0 U3g(Build 24853646).

Workaround:

To address this issue, stop the vc-ws1a-broker service and disable vc-ws1a-broker if external IDP such as Azure AD and Okta is not used.

vmon-cli --stop vc-ws1a-broker vmon-cli --update vc-ws1a-broker --starttype DISABLE