VCF Operations component displays Powered Off in VMware Cloud Foundation 9.1 due to service account complexity constraints

Products

• VCF Operations

Issue/Introduction

In VMware Cloud Foundation (VCF) 9.1, the VCF Operations component may incorrectly report a status of Powered Off within the VCF Management UI under Build > Lifecycle > VCF Management > Components > VCF Operations.

While the appliance remains fully functional and accessible via its admin console (https://<vcfops_ip>/admin), the Actions dropdown menu is missing in the SDDC Manager interface. Technical analysis indicates that the sddc-build-service cannot acquire an authentication token for the lcm-ops service account, resulting in the following errors:

• casa.log: Credentials for user 'lcm-ops' don't exist or are empty.
• Suite-API Error: {"type":"Error","message":"The provided username/password or token is not valid. Please try again.","httpStatusCode":401,"apiErrorCode":401}
• API Error Message: New password does not meet complexity requirements. (HTTP 500)

Environment

VMware Cloud Foundation 9.1
VCF Operations 9.1

Cause

The global password complexity policies are incorrectly enforced on internal service accounts. As a result, the lcm-ops account creation fails because its auto-generated password does not meet the complexity requirements (such as minimum length) dictated by the CASA authentication API.

Resolution

To resolve this issue, you must temporarily relax the password complexity policy and manually provision the missing service account.

Prerequisites:

• Take a snapshot of the VCF Operations nodes. Reference Doc
• Ensure root access to the VSP Control Plane VM is available.

Step 1: Relax the password policy Execute the following curl command to adjust the complexity parameters via the Suite API:

curl -k --request PUT \
  --url https:///suite-api/internal/passwordmanagement/ops/policy \
  --header 'authorization: OpsToken ' \
  --header 'content-type: application/json' \
  --header 'x-ops-api-use-unsupported: true' \
  --data '{"complexityConstraint":{"minLength":8,"minLowercase":1,"minUppercase":1,"minNumeric":1,"minSpecial":1,"passwordHistory":1},"expirationConstraint":{"passwordExpirationDays":180},"lockoutConstraint":{"lockoutMaxAuthFailures":3,"lockoutEvaluationPeriod":300,"lockoutPeriod":600}}'

Step 2: Retrieve the intended lcm-ops credentials Log in to the VSP Control Plane VM and run the following commands to extract the generated password from the Kubernetes secret:

• export KUBECONFIG=/etc/kubernetes/admin.conf
• kubectl -n vcf-sddc-lcm get secret -l vcf.broadcom.com/component-type=ops -o jsonpath='{.items[0].data.clientSecret}' | base64 -d; echo

Step 3: Manually create the service account Use the password retrieved in Step 2 to create the account via the CASA API:

curl -sk -u ':' \
  -X POST https:///casa/auth/users \
  -H 'Content-Type: application/json' \
  -d '{"username":"lcm-ops","password":"","key":"ops-lcm"}' \
  -w '\nHTTP=%{http_code}\n'

Verification: A successful response returns an HTTP 200 status and a valid token. After the next synchronization cycle, the VCF Operations status will restore to Active or Online, and the Actions menu will become available.

Note: It is strongly recommended to maintain the minimum password length parameter at a maximum of 20 characters until the Environment is upgraded to 9.1.1. This architectural limitation has been successfully addressed in the upcoming VCF 9.1.x release, wherein standard user password policies will no longer govern internal service accounts