"error instantiating a new VM: API Error: 400: VDC Compute Policy VM_PLACEMENT_POLICY_NAME is not available in VDC ORG_VDC_NAME" error stops Provider level users deploying TKG clusters through Container Service Extension

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• Attempting to create a Tanzu Kubernetes Grid (TKG) cluster fails in Cloud Director through Container Service Extension as a Provider level user such as a System Administrator.
• The vApp and initial Ephemeral VM for the TKG cluster are created but the first Control Plane node or Worker node VM is never created.
• A VM Placement Policy is selected for the Control Plane and/or Worker nodes during TKG cluster creation.
• The Cluster API Provider for VMware Cloud Director (CAPVCD) logs on the Ephemeral VM show an error adding the VMs to the vApp for the TKG cluster using the Cloud Director action/recomposeVApp API:

error instantiating a new VM: API Error: 400: VDC Compute Policy <VM_PLACEMENT_POLICY_NAME> is not available in VDC <ORG_VDC>

• Creating VMs in the Cloud Director UI or API succeeds when using the same VM Placement Policy, in the same Organization VDC, as the same Provider level user.

Environment

VMware Cloud Director 10.x

Cause

This is a known issue affecting Container Server Extension deployments configured to use Cluster API Provider for VMware Cloud Director (CAPVCD) v1.3.0 or earlier when multiple VM Placement Policies with the same name exist in the Cloud Director instance.

Resolution

This issue will be resolved in a future release of Cluster API Provider for VMware Cloud Director (CAPVCD) after v1.3.0.
The available versions of CAPVCD can be seen on the component's GitHub page here, Releases.

To update Container Service Extension to use a newer CAPVCD release when available the following documentation can be followed, Update Server Configuration.

Workaround:
To workaround the issue ensure that all VM Placement Policies in Cloud Director have unique names if they are to be used for TKG cluster creation.
The name of an existing VM Placement Policy can be edited as per the Cloud Director documentation here, Edit a VM Placement Policy in VMware Cloud Director.

Additional Information

For information on how to generate logs from Container Service Extension and an Ephemeral VM (Bootstrap VM) see the documentation, Troubleshooting - Log Analysis from Bootstrap VM.