vCenter Server displays alarm: "vc-ws1a-broker status changed from green to unknown"

Products

• VMware vCenter Server

Issue/Introduction

• The following alarm is reported: "vc-ws1a-broker status changed from green to unknown".
• The status of vc-ws1a-broker returns to green approximately 5 minutes later.
• When looking at the system journal using the command "journalctl -b 0" the following events can be seen:

  Apr 02 18:11:44 <vCenter_name> vpxd[10740]: Event [85445] [1-1] [2025-04-02T18:11:44.187012Z] [vim.event.HealthStatusChangedEvent] [info] [Vmonuser] [] [85445] [vc-ws1a-broker status changed from green to unknown]
  Apr 02 18:11:44 <vCenter_name> vpxd[10740]: Event [85446] [1-1] [2025-04-02T18:11:44.193412Z] [vim.event.EventEx] [info] [] [] [85446] [Alarm 'Health status changed alarm' on Datacenters triggered by event 85445 'vc-ws1a-broker status changed from green to unknown']
  Apr 02 18:16:50 <vCenter_name> vpxd[10740]: Event [85457] [1-1] [2025-04-02T18:16:50.570414Z] [vim.event.HealthStatusChangedEvent] [info] [Vmonuser] [] [85457] [vc-ws1a-broker status changed from unknown to green]
  Apr 02 18:16:50 <vCenter_name> vpxd[10740]: Event [85458] [1-1] [2025-04-02T18:16:50.571736Z] [vim.event.EventEx] [info] [] [] [85458] [Alarm 'Health status changed alarm' on Datacenters triggered by event 85457 'vc-ws1a-broker status changed from unknown to green']

• Additionally, /var/log/vmware/vc-ws1a-broker/nginx-error.log contains errors like below:

  2025/04/02 18:11:42 [alert] 67#0: *1874 pread() read only 0 of 5 from "/opt/vmware/idm/health/healthCheck" while sending response to client, client: ::ffff:127.0.0.1, server: localhost, request: "GET /sddc/broker/health HTTP/1.1", host: "localhost:10121"

• Normally this issue does not impact on vCenter operation but if VCHA configured env, failover can happened due to this issue.
  (In /var/log/vmware/vmon/vmon.log)
  

  [DATE/TIME] In(05) host-1938 <vc-ws1a-broker-healthcmd> Constructed command: /storage/containers/vc-ws1a-broker/dbb7fed8d494caf9bf2ee657a3622d62b846347019b4e13df5f0bd59128010c0/health_check.sh
  [DATE/TIME] Wa(03) host-1938 <vc-ws1a-broker> Service api healthcheck command returned unknown exit code 1
  [DATE/TIME] Er(02) host-1938 <vc-ws1a-broker> health state unknown ,considered as system failure
  [DATE/TIME] Er(02) host-1938 System Failure, initiating system restart.

Environment

vCenter Server 8.0

Cause

The vc-ws1a-broker container may rarely fail due to a read error when accessing health check data.

Resolution

The issue resolved in vCenter 8.0 Update 3g.

 

Additional Information

https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/release-notes/vcenter-server-update-and-patch-release-notes/vsphere-vcenter-server-80u3g-release-notes.html

vSphere High Availability Issues
PR 3507227: A faulty health status of a microservice under VMware Identity Services might trigger an unexpected vCenter High Availability failover
A false negative health status of a microservice under VMware Identity Services that occurs in very specific conditions might trigger an unexpected vCenter High Availability failover.