TCA-2.3/3.0 TKG nodes are unable to resolve hostnames with the ".local" domain suffix
Article ID: 345726
Updated On:
Products
VMware
VMware Telco Cloud Automation
Issue/Introduction
Symptoms:
- Attempts to resolve hostnames that have a domain suffix that ends in ".local" fail on Tanzu Kubernetes Grid (TKG) nodes.
- Attempts to resolve hostnames that have a domain suffix that ends in ".local" fail on Tanzu Kubernetes Grid (TKG) nodes.
Environment
VMware Telco Cloud Automation 2.3
Cause
A known issue is documented in Tanzu Kubernetes Grid (TKG) as per VMware KB: https://kb.vmware.com/s/article/83623?lang=en_US
Resolution
This is a known issue affecting Tanzu Kubernetes Grid. There is currently no resolution in TCA 2.3/TCA3.0.
Workaround:
To deploy TKG Management Cluster with .local domain from TCA 2.3/ 3.0 UI:
Step 1:
A. For TCA 2.3 - SSH to TCA-CP with Admin credentials > Switch to root user > and then proceed with Step C:
B. For TCA 3.0 SSH to TCA-CP with Admin credentials and perform the following commands then proceed with step C:
D. Create below two files (vsphere-overlay-dns-control-plane.yaml | vsphere-overlay-dns-workers.yaml) with the content in the directory “/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/”
vi vsphere-overlay-dns-control-plane.yaml
vi vsphere-overlay-dns-workers.yaml
Step2: Deploy the TKG Management cluster from TCA UI.
Step 3: After TKG Management cluster creation execute the below steps before starting v2 TKG Workload Cluster.
I: SSH to the TKG Management cluster with CAPV user
II: Identify the "tca-kubecluster-operator" pod:
III: Create the above same files with same content in the management cluster master node and copy it to the tca-kubecluster-operator-6879b6bd48-fv697 under “/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/” as mentioned in the above Step 1-D.
Example:
IV: Login to the POD and validate the files:
Step 4: Now proceed with the v2 TKG workload cluster deployment.
Workaround:
To deploy TKG Management Cluster with .local domain from TCA 2.3/ 3.0 UI:
Step 1:
A. For TCA 2.3 - SSH to TCA-CP with Admin credentials > Switch to root user > and then proceed with Step C:
B. For TCA 3.0 SSH to TCA-CP with Admin credentials and perform the following commands then proceed with step C:
admin@tca [ ~ ]$ kubectl get pods -A|grep tkg230 tca-cp-cn kbs-tkg230-55795ccb89-jlrdz 1/1 Running 2 (8d ago) 28d admin@tca [ ~ ]$ kubectl exec -n tca-cp-cn kbs-tkg230-55795ccb89-jlrdz -it bash kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. Defaulted container "kbs-tkg230-operator" out of: kbs-tkg230-operator, init-bootstrapper (init)
C. Update the file “config_default.yaml” for the below parameters with your .local domain under the directory “/root/.config/tanzu/tkg/providers/”.
#! Specifies the service domain for the cluster SERVICE_DOMAIN: #! Node SearchDomains CONTROL_PLANE_NODE_SEARCH_DOMAINS: vmware.local WORKER_NODE_SEARCH_DOMAINS: vmware.local
D. Create below two files (vsphere-overlay-dns-control-plane.yaml | vsphere-overlay-dns-workers.yaml) with the content in the directory “/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/”
vi vsphere-overlay-dns-control-plane.yaml
#@ load("@ytt:overlay", "overlay")
#@ load("@ytt:data", "data")
#@overlay/match by=overlay.subset({"kind":"VSphereMachineTemplate", "metadata": {"name": data.values.CLUSTER_NAME+"-control-plane"}})
---
spec:
template:
spec:
network:
devices:
#@overlay/match by=overlay.all, expects="1+"
-
#@overlay/match missing_ok=True
nameservers: ["172.22.22.43"]
#@overlay/match missing_ok=True
searchDomains: ["vmware.local"]
vi vsphere-overlay-dns-workers.yaml
#@ load("@ytt:overlay", "overlay")
#@ load("@ytt:data", "data")
#@overlay/match by=overlay.subset({"kind":"VSphereMachineTemplate", "metadata": {"name": data.values.CLUSTER_NAME+"-worker"}})
---
spec:
template:
spec:
network:
devices:
#@overlay/match by=overlay.all, expects="1+"
-
#@overlay/match missing_ok=True
nameservers: ["172.22.22.43"]
#@overlay/match missing_ok=True
searchDomains: ["vmware.local"]
Step2: Deploy the TKG Management cluster from TCA UI.
Step 3: After TKG Management cluster creation execute the below steps before starting v2 TKG Workload Cluster.
I: SSH to the TKG Management cluster with CAPV user
II: Identify the "tca-kubecluster-operator" pod:
capv@tcpran30-mgmt-9fqh5-k8fkt [ ~ ]$ kubectl get pods -n tca-system NAME READY STATUS RESTARTS AGE nodeconfig-daemon-97csj 1/1 Running 0 121m nodeconfig-daemon-control-plane-4lhtw 1/1 Running 0 121m nodeconfig-daemon-control-plane-h9s5c 1/1 Running 0 121m nodeconfig-daemon-control-plane-s8w6b 1/1 Running 0 121m nodeconfig-daemon-fcjpj 1/1 Running 0 121m nodeconfig-operator-7fcd9b44b8-qtvff 2/2 Running 0 121m tca-diagnosis-operator-548d99df66-8276c 1/1 Running 0 120m tca-kubecluster-operator-6879b6bd48-fv697 1/1 Running 0 120m test-controller-65d68cf67-9lqbn 1/1 Running 0 121m vmconfig-operator-bfd9c589-2qrf7 2/2 Running 2 (119m ago) 119m
III: Create the above same files with same content in the management cluster master node and copy it to the tca-kubecluster-operator-6879b6bd48-fv697 under “/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/” as mentioned in the above Step 1-D.
Example:
capv@tcpran30-mgmt-9fqh5-k8fkt [ ~ ]$ kubectl -n tca-system cp vsphere-overlay-dns-control-plane.yaml tca-kubecluster-operator-6879b6bd48-fv697:/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/
capv@tcpran30-mgmt-9fqh5-k8fkt [ ~ ]$ kubectl -n tca-system cp vsphere-overlay-dns-workers.yaml tca-kubecluster-operator-6879b6bd48-fv697:/root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt/
IV: Login to the POD and validate the files:
capv@tcpran30-mgmt-9fqh5-k8fkt [ ~ ]$ kubectl exec tca-kubecluster-operator-6879b6bd48-fv697 -n tca-system -it bash NOTE: kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. root [ / ]# ls -al /root/.config/tanzu/tkg/providers/infrastructure-vsphere/ytt total 32 drwxr-xr-x 1 root root 4096 2023-05-08 10:01 . drwxr-xr-x 1 root root 4096 2023-03-09 09:50 .. -rw-r----- 1 1000 users 491 2023-05-08 10:00 vsphere-overlay-dns-control-plane.yaml -rw------- 1 root root 8392 2023-03-09 09:50 vsphere-overlay.yaml -rw-r----- 1 1000 users 484 2023-05-08 10:01 vsphere-overlay-dns-workers.yamlV: At POD - Update the file “config_default.yaml” for the below parameters with your .local domain under the directory “/root/.config/tanzu/tkg/providers/”:
# cd /root/.config/tanzu/tkg/providers/ root [ ~/.config/tanzu/tkg/providers ]# cat config_default.yaml | grep NODE_SEARCH_DOMAINS CONTROL_PLANE_NODE_SEARCH_DOMAINS: WORKER_NODE_SEARCH_DOMAINS: root [ ~/.config/tanzu/tkg/providers ]# sed -i "s/CONTROL_PLANE_NODE_SEARCH_DOMAINS:/CONTROL_PLANE_NODE_SEARCH_DOMAINS: vmware.local/g" config_default.yaml root [ ~/.config/tanzu/tkg/providers ]# sed -i "s/WORKER_NODE_SEARCH_DOMAINS:/WORKER_NODE_SEARCH_DOMAINS: vmware.local/g" config_default.yaml root [ ~/.config/tanzu/tkg/providers ]# cat config_default.yaml | grep NODE_SEARCH_DOMAINS CONTROL_PLANE_NODE_SEARCH_DOMAINS: vmware.local WORKER_NODE_SEARCH_DOMAINS: vmware.local
Step 4: Now proceed with the v2 TKG workload cluster deployment.
Additional Information
Impact/Risks:
Failed to deploy TKG Management or Workload clusters when you have hostnames with the ".local" domain suffix from TelcoCloudAutomation (TCA)-UI.
Failed to deploy TKG Management or Workload clusters when you have hostnames with the ".local" domain suffix from TelcoCloudAutomation (TCA)-UI.
Feedback
Yes
No
Powered by
