VCF Operations cluster deployment blocked when Continuous Availability is enabled

Products

VCF Operations

Issue/Introduction

VCF installation (converge) using brownfield vSphere deployments with VCF Operations, where continuous availability is enabled for the VCF Operations cluster, fails with the following error:

"The Fleet lifecycle 'Update component' task with ID ######-####-####-####-############ failed. Internal errors: Task with ID '######-####-####-####-############' from SDDC lifecycle with ID '######-####-####-####-############' has completed with failure(s). Check the errors in the next sub-task(s) for more details., Adding nodes to VCF Operations cluster is not allowed when Continuous Availability (CA) is enabled. Disable CA on the VCF Operations cluster before adding new nodes., Adding nodes to VCF Operations cluster is not allowed when Continuous Availability (CA) is enabled. Disable CA on the VCF Operations cluster before adding new nodes. Reference Token: 6VK0ES"

Note: The reference token will change.

Environment

VMware Cloud Foundation 9.1.x

Cause

VCF Operations cluster expansion is blocked in the Fleet Lifecycle component, as the expansion will result in the cluster degrading and some of the VCF operations functionalities will be impacted.
During VCF installation (converge), the installer will only import the VCF operations cluster details to Fleet LCM; it will not deploy new VCF operations nodes. But cloud proxy also follows the same deployment workflow and tasks; it is incorrectly blocking the deployment, assuming it is VCF operations clusters node deployment.
Ideally the guardrail is not applicable during the VCF installation; it is applicable only for the DayN VCF operations cluster expansion operation.

Resolution

The fix will be available in future releases.

Workaround:

SSH into one of the VMware Cloud Foundation (VCF) Service Runtime control nodes using vmware-system-user (the password is the same as the VCF Service Runtime system user password).
Run the following command to switch to the root/sudo environment:
```
sudo su
export KUBECONFIG=/etc/kubernetes/admin.conf
```
Run the following command to edit the ConfigMap and add the property to skip CA validation:
```
kubectl edit cm vcf-sddc-build-service-sddcbuild-application-properties -n vcf-sddc-lcm
```
Manually edit the properties by changing the following:
```
data:
  application.properties: ""
```
to:
```
data:
  application.properties: |
    ops.ca.validation.disabled=true
```
Note: If application.properties is not empty and contains other properties, insert ops.ca.validation.disabled=true on a new line.
Save and close the file.

Restart the SDDC build pod:

kubectl rollout restart deployment -n vcf-sddc-lcm -l app=sddcbuild

Wait a few minutes and confirm that the SDDC build service pod is running:
```
kubectl get pods -n vcf-sddc-lcm -l app=sddcbuild
```
Retry the VCF installation from the Installer UI.
After successful VCF installation, revert the property changes and restart the SDDC build service to block Day-N scaling operations from Fleet LCM:
```
kubectl edit cm vcf-sddc-build-service-sddcbuild-application-properties -n vcf-sddc-lcm
```
Revert the configuration to:
```
data:
  application.properties: ""
```
Save and close the file.