Symptoms:
- Installing VMware Cloud Director Extension for VMware Tanzu Mission Control through Cloud Director Solution Add-On Management.
- Pods in the tmc-local namespace on the target Kubernetes cluster have a CreateContainerConfigError status, for example:
NAMESPACE NAME READY STATUS
tmc-local audit-service-consumer-<POD_ID> 0/1 CreateContainerConfigError
tmc-local audit-service-consumer-<POD_ID> 0/1 CreateContainerConfigError
tmc-local audit-service-server-<POD_ID> 0/1 CreateContainerConfigError
tmc-local audit-service-server-<POD_ID> 0/1 CreateContainerConfigError
tmc-local dataprotection-server-<POD_ID> 0/1 CreateContainerConfigError
tmc-local dataprotection-server-<POD_ID> 0/1 CreateContainerConfigError
tmc-local inspection-server-<POD_ID> 0/2 CreateContainerConfigError
tmc-local inspection-server-<POD_ID> 0/2 CreateContainerConfigError
- Pods in the tmc-local namespace on the target Kubernetes cluster have a CreateContainerConfigError status and show events regarding missing secrets, for example the audit-service-consumer-<POD_ID> pods show events similar to the following:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount <AGE> kubelet MountVolume.SetUp failed for volume "kafka-root-ca" : secret "audit-server-kafka-creds" not found
Warning FailedMount <AGE> kubelet MountVolume.SetUp failed for volume "consumer-tls" : secret "audit-consumer-tls" not found
Warning Failed <AGE> kubelet Error: secret "audit-s3-creds" not found
- The container logs on the s3-access-operator-<POD_ID> pod in the tmc-local namespace has errors of the form:
<TIME_STAMP> [30mERROR[0m Reconciler error {"controller": "s3accesspolicy", "controllerGroup": "infra.tmc.eng.vmware.com", "controllerKind": "S3AccessPolicy", "S3AccessPolicy": {"name":"inspection-inspection-s3","namespace":"tmc-local"}, "namespace": "tmc-local", "name": "inspection-inspection-s3", "reconcileID": "<RECONCILE_ID>", "error": "error in retrieving credentials value: RequestError: send request failed\ncaused by: Post \"https://<TMC_DNS_ZONE>/\": dial tcp: lookup <TMC_DNS_ZONE> on <DNS_IP>:53: no such host"}