Registration Failure with HTTP 530 Error during SALT_RAAS Component Registration in VCF Operations 9.1

Products

VMware SDDC Manager / VCF Installer VCF Operations

Issue/Introduction

While re-registering components in VCF Operations 9.1 for the primary brownfield instance, the process fails on the SALT_RAAS component.

Both the Fleet Build Service (FBS) and SDDC Build Service (SBS) detect that the SALT_RAAS component is missing during registration and attempt to automatically create a service account.
The creation request fails, resulting in an HTTP 500 error in the Fleet Build Service and an HTTP 530 Unknown status code from the Salt RaaS Remote Procedure Call (RPC) endpoint (/raas/rpc).

From Fleet Build Service logs:

YYYY-MM-DDTHH:MM:SS stdout F     id: com.broadcom.vcf.lcm.SaltRaasClientError
YYYY-MM-DDTHH:MM:SS stdout F     defaultMessage: Unexpected Salt RaaS client error has occurred.
YYYY-MM-DDTHH:MM:SS stdout F     localizedMessage: Unexpected Salt RaaS client error has occurred.
YYYY-MM-DDTHH:MM:SS stdout F     args: {}
YYYY-MM-DDTHH:MM:SS stdout F }
YYYY-MM-DDTHH:MM:SS stdout F YYYY-MM-DDTHH:MM:SS732Z ERROR Fleet Build 44 [lcm@4413 scPath="ops" thread="system-task-worker-4" logger="c.b.v.l.f.b.o.a.RegisterComponentsInOpsPlatformAction" method="lambda$execute$0:L"] [#######-####-###-###-#####, taskId=#######-####-###-###-#####] - Unable to register component with Id: #######-####-###-###-##### in VCF Ops.
YYYY-MM-DDTHH:MM:SS stdout F com.broadcom.vcf.lcm.fleet.build.exception.SddcLcmClientException: API response for operation 'create service account' is not a successful one.
YYYY-MM-DDTHH:MM:SS stdout F         at com.broadcom.vcf.lcm.fleet.build.service.util.SddcLcmClientServiceUtil.executeSyncApiRequestAndGetResponse(SddcLcmClientServiceUtil.java:190)
<SNIP>

From SDDC Build Service logs:

POST https://<vcf-fqdn>:443/raas/rpc
Headers: [Accept:"application/json", Content-Type:"application/json", ...]
Body: {"resource":"auth","method":"save_user","kwarg":{"config_name":"internal","username":"svc-opssr-salt_raas","roles":["Superuser"]}}

<<< HTTP Response <<<
Code: 530 Unknown (23ms)
ERROR sddc-build-service [...] Failed to create service account for Salt Raas for user svc-opssr-salt_raas

From /var/log/salt/minion logs:

YYYY-MM-DDTHH:MM:SS.msZ ERROR salt-minion 353987 [salt@4413] salt.crypt: The Salt Master has cached the public key for this node, this salt minion will wait for 10 seconds before attempting to re-authenticate YYYY-MM-DDTHH:MM:SS.msZ ERROR salt-minion 353987 [salt@4413] salt.minion: Error dispatching event. Message timed out

Environment

VMware Cloud Foundation (VCF) 9.1
VCF Operations 9.1

Cause

Salt RaaS relies on a local Redis cache to validate entitlements and process API calls (such as service account creation). During startup, RaaS validates entitlements with VCF Operations and populates both its database and the Redis cache.

If the Redis license cache drops or becomes stale during operation, RaaS does not have a fallback mechanism to dynamically query VCF Operations on incoming API requests. Because the cache entry is missing, RaaS rejects the request with an HTTP 530 Unknown error.

Resolution

Direct modification of database tables is required. Open a support ticket with Broadcom Support for assisted resolution.

Additional Information

To investigate and monitor container health on the VCF Services Platform (VSP / VMSP):

Open an SSH session to the target VCF Services Platform control plane node using vmware-system-user account, then escalate privileges to root:
```
su -
```
Set the Kubernetes configuration context to gain administrative access to cluster management tools:
```
export KUBECONFIG=/etc/kubernetes/admin.conf
```
Check the status of all cluster pods to locate any failing, crashing, or stuck components:
```
kubectl get pods -A
```
Stream real time logs for the specific pod experiencing issues:
```
kubectl logs -f <POD_NAME> -n <NAMESPACE>
```