"[error while enabling cloudinit on the machine [xxxx/EPHEMERAL_TEMP_VM]; unable to set vm extra config key [guestinfo.userdata.encoding] for vm" failure when attempting to create a Tanzu Kubernetes Grid Cluster using Cloud Director Container Service Exte

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• Attempting to create a Kubernetes Cluster using Cloud Director Container Service Extension fails immediately and enters an Error state.
• Ephemeral VM, EPHEMERAL_TEMP_VM, gets created but stays in powered off state.
• Creation of cluster fails when user logs in as a Tenant user such as Organization Administrator but the same functionality works as a System Administrator.
• The Events on the Kubernetes Cluster show a ScriptInitError error with the following detail:

[error while enabling cloudinit on the machine [cxxxx/EPHEMERAL_TEMP_VM]; unable to set vm extra config key [guestinfo.userdata.encoding] for vm : [status code is [400], response body: [<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <Error xmlns="http://www.vmware.com/vcloud/v1.5" xmlns:vmext="http://www.vmware.com/vcloud/extension/v1.5" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:common="http://schemas.dmtf.org/wbem/wscim/1/common" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vmw="http://www.vmware.com/schema/ovf" xmlns:ovfenv="http://schemas.dmtf.org/ovf/environment/1" xmlns:ns9="http://www.vmware.com/vcloud/versions" message="Missing right to use ExtraConfig guestinfo.userdata.encoding." majorErrorCode="400" minorErrorCode="BAD_REQUEST"/> ]]] during cluster creation

• The /root/cse.log on the CSE Server will have the following entries:

{"level":"error","ts":"202x-xx-xxTxx:xx:xx.xxxZ","caller":"app/main.go:533","msg":"error creating cluster [xxxx(urn:vcloud:entity:vmware:capvcdCluster:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)] : [error setting cloud init script for cluster [xxxx(urn:vcloud:entity:vmware:capvcdCluster:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)], [error while enabling cloudinit on the machine [xxxx/EPHEMERAL_TEMP_VM]; unable to set vm extra config key [guestinfo.userdata.encoding] for vm : [status code is [400], response body: [<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error xmlns=\"http://www.vmware.com/vcloud/v1.5\" xmlns:vmext=\"http://www.vmware.com/vcloud/extension/v1.5\" xmlns:ovf=\"http://schemas.dmtf.org/ovf/envelope/1\" xmlns:vssd=\"http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData\" xmlns:common=\"http://schemas.dmtf.org/wbem/wscim/1/common\" xmlns:rasd=\"http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData\" xmlns:vmw=\"http://www.vmware.com/schema/ovf\" xmlns:ovfenv=\"http://schemas.dmtf.org/ovf/environment/1\" xmlns:ns9=\"http://www.vmware.com/vcloud/versions\" message=\"Missing right to use ExtraConfig guestinfo.userdata.encoding.\" majorErrorCode=\"400\" minorErrorCode=\"BAD_REQUEST\"/>\n]]]]","stacktrace":"main.processRDE\n\t/app/main.go:533"}

Environment

VMware Cloud Director for Service Provider 10.x

Cause

The Kubernetes Clusters deployed by Container Service Extension require VMs to be created in Cloud Director with ExtraConfig elements.

The user creating Kubernetes Clusters must have a Global Role with the necessary Rights to create the VMs with these elements including Preserve All ExtraConfig Elements During OVF Import and Export.

The Organization where Kubernetes Clusters are created must also have a Rights Bundle with the necessary Rights published to it also including Preserve All ExtraConfig Elements During OVF Import and Export.

Resolution

To resolve this issue ensure both the Global Roles used by the Tenant users deploying Kubernetes Clusters and the Rights Bundle published to the Tenant Organizations have the Right Preserve All ExtraConfig Elements During OVF Import and Export enabled.

To enable the Right Preserve All ExtraConfig Elements During OVF Import and Export for the Global Roles used by the Tenant users deploying Kubernetes Clusters perform the following steps:

1. Log into the Cloud Director Provider UI as a System Administrator and navigate to Administration, Tenant Access Control, Global Roles.
2. Select the Global Role to be used by Tenant users to deploy Kubernetes Clusters and click Edit.
3. Expand the section Compute, vApp and enable the Right Preserve All ExtraConfig Elements During OVF Import and Export.
4. Save the changes and Publish the Global Role to the desired Tenant Organizations.

To enable the Right Preserve All ExtraConfig Elements During OVF Import and Export for the Rights Bundle published to the Tenant Organizations perform the following steps:

1. Log into the Cloud Director Provider UI as a System Administrator and navigate to Administration, Tenant Access Control, Rights Bundles.
2. Select the Kubernetes Clusters Rights Bundle and click Edit.
3. Expand the section Compute, vApp and ensure the Right Preserve All ExtraConfig Elements During OVF Import and Export is enabled.
4. Save the changes and Publish the Rights Bundle to the desired Tenant Organizations.

Additional Information

For more information on the Roles and Rights required for Tenant users to manage Kubernetes Clusters see VMware Cloud Director Container Service Extension Product Documentation, Managing Tenant User Roles and Rights.