Kubernetes Policies in VCD 10.2 with vCenter 7.0 Tanzu are non-functional

Products

• VMware Cloud Director

Issue/Introduction

In order to utilize Tanzu Kubernetes in VCD, the ability to create and publish Kubernetes policies is required, which is not possible without specifying Virtual Machine Classes.

Upon creating a Provider Virtual Data Center (PVDC) backed by a Tanzu Kubernetes Supervisor Cluster in vCenter Server, VMware Cloud Director (VCD) will display a Kubernetes icon next to the PVDC. However, an autogenerated Kubernetes PVDC policy will not be created. Additionally, any attempts to publish or create a new Kubernetes PVDC policy to a Virtual Data Center (VDC) fail because no Virtual Machine Classes are present to select from.
There is an audit event detailing the failure with the message:

Cloud Director cannot reach vSphere for Kubernetes, reason message: Could not connect to vSphere for Kubernetes infrastructure

Environment

• This currently applies to VMware Cloud Director 10.2 and 10.2.1

Cause

Due to the certificate structure of Tanzu Kubernetes in vCenter, the certificate of the Supervisor Cluster is not automatically trusted by VCD. Calls made to the Supervisor Cluster by VCD fail due to SSL errors.

 

 

 

Resolution

Note: For VMware Cloud Director 10.2.2 use KB Kubernetes Policies in VCD 10.2 with vCenter 7.0 Tanzu are non-functional 

The Supervisor Cluster certificate can be manually trusted and accepted by VCD. The steps to manually set the certificate to trusted are:

1. In the vCenter UI, navigate to Menu > Developer Center > API Explorer > GET /api/vcenter/namespace-management/clusters/{cluster} > Try it out
2. In the value for the cluster parameter, input the moref of the vCenter Cluster containing the Kubernetes Supervisor Cluster. (The moref is located in the URL of the h5 UI when clicking on the cluster. It is in the format: domain-##
3. Click execute to execute the API and click on vcenter.namespace_management.clusters.info to expand the response. Make note of the IP under api_server_management_endpoint, this is the Control Plane IP endpoint.
4. Via postman, do a POST on VCD's testConnection endpoint (POST https://{IP}/cloudapi/1.0.0/testConnection) with the Control Plane IP address. The payload should be:

{
    "secure": true,
    "host": "{Control Plane IP from step 3}",
    "port": 443
}

Capture the certificate in the response.

5. Via postman, do a POST against vCD's trustedCertificates endpoint (POST https://{IP}/cloudapi/1.0.0/ssl/trustedCertificates) with the following payload:

{
   alias: {someAlias},
   certificate: {certificate captured from step 4}
}

6. In the VCD provider UI, navigate to the Tanzu Kubernetes vCenter. Go to Infrastructure Resources > vCenter Server Instances > {specific vCenter). Click on the vCenter Server.
7. Click Reconnect to reconnect the vCenter Server.
8. Once reconnection is complete, navigate to the Kubernetes-enabled PVDC and verify that an autogenerated policy is now present with Virtual Machine Classes. (Cloud Resources > Provider VDCs > {Kubernetes-enabled PVDC} > Policies > Kubernetes)

Note: As an alternative to steps 4 and 5, you can navigate to https://{control plane IP} in a web browser and save the certificate to a local file. This file can then be uploaded to VCD via the Provider UI: Administration > Certificate Management > Trusted Certificates > Import .