Error: Identity broker is unhealthy. View additional details in the 'Alerts' section under 'Infrastructure Operations' when configuring VCF SSO

Products

• VCF Operations

Issue/Introduction

• In the VCF Operations UI, under the Identity and Access tab, the VCF Identity Broker (vIDB) displays as unhealthy with the following error message: Identity broker is unhealthy. View additional details in the 'Alerts' section under 'Infrastructure Operations'.


• Attempts to configure VCF SSO stall at Step 1, displaying an "identity broker appliance was not available" message.
  
  
• Aria Operations nodes remain stuck waiting for analytics state. The Aria Operations analytics log (/storage/log/vcops/analytics.log) indicates shard purge failures due to offline servers. Following KB 423584 to reset internal node coordination by taking the cluster offline in the Admin UI, rebooting all nodes, and bringing the cluster back online does not resolve the issue.
  
  ERROR analytics 25905 [ops@#### threadId="######" threadName="pool-##-thread-#"] [com. vmware. vcops.platform. common. sharding. ShardingManagerClient.purgeResource] - purgeResource failed on shard ########-####-#####-####-########## : The requested server (s) are not running
org. apache. geode. cache. execute. FunctionException: The requested server (s) are not running
at com. vmware. vcops.platform.gemfire.GemfireFunctionExecutor. onMemberWithTimeOut (GemfireFunctionExecutor. java: ###) ~[alive_platform. jar : 
....
Caused by: org. apache. geode. cache. execute. FunctionInvocationTargetException: The requested server(s) are not running
... 11 more

• Execution of kubectl -n vidb-external get pods reveals several pods in CrashLoopBackOff and error states.
  
  

• Execution of kubectl -n vidb-external get sts confirms the Postgres StatefulSet remains READY below the desired replica count indefinitely without autonomous recovery.

  root@<NodeFQDN>[ ~ ]# kubect1 -n vidb-external get sts

  NAME                               READY      AGE

  vidb-postgres-instance             0/3        47d

• Execution of kubectl exec <PostgresPodName> -n prelude -- patronictl list confirms that the leader node is lost and pods are stuck in 'starting' state
  
  

• Execution of kubectl -n vidb-external get events displays repeating Unhealthy warnings for the Postgres container: Readiness probe failed: HTTP probe failed with statuscode: 503.

• Execution of kubectl -n vidb-external logs <vidb-postgres-pod-log> loops indefinitely with:
  
  ... INFO: Still starting up as a standby.
... WARNING: Failed to determine PostgreSQL state from the connection, falling back to cached role
... INFO: Error communicating with PostgreSQL. Will try again later
ERROR: ... Archive '<wal_segment>' does not exist.
ERROR: ... Archive '<tli>.history' does not exist.

Environment

VMware Cloud Foundation Operations 9.0.x
VMware Cloud Foundation Identity Broker (vIDB) 9.0.x

Cause

The deadlock is caused by the Kubernetes OrderedReady policy, which refuses to replace the missing leader pod because surviving replicas are permanently stuck in a NotReady state (due to missing WAL archives). This prevents any new Patroni leader election. 

Resolution

To resolve this issue, please contact Broadcom Support and note this Article ID 449494 in the problem description.