Unable to connect to the server: dial tcp X.X.X.X:6443: i/o timeout

Products

• VMware NSX

Issue/Introduction

 

• Fetching cluster, pod, or information of any using kubectl commands is failing intermittently with the following message. 'unable to connect to the server: dial tcp X.X.X.X:6443: i/o timeout.
• kubectl get pvc -n namespace
• kubectl get po -n namespace
• kube api server IP 10.20.8.5 port 6443.
• Tier-1 uplink, we could see the SYN was reaching the virtual server, but no response was received.
• The connection table shows the connection to the "pool member that is not part of".

• Connection works when hitting the actual pool member.

• NSX Manager UI shows the right supervisor control plane vm pool members of the LB pool.
• The CLI output of the pool member from the edge shows the non-existing supervisor control plane vm IP as the pool member.

Source - 10.10.10.142
Virtual Server - 10.20.8.5
Non-existing pool member - Y.Y.4.16

Actual pool member  - Y.Y.4.12.

Environment

VMware NSX

Cause

The configuration has not been realized from the Manager to the edge.

Resolution

Workaround

 Restarting the Proton service or performing a rolling reboot of the NSX Manager.

Additional Information

If the service restart or rolling reboot of the manager hasn't resolved the issue, kindly raise a support case with this KB reference.

Handling Log Bundles for offline review with Broadcom support.

• Collect Support Bundles for Troubleshooting NSX-T
• Uploading Diagnostic information on Broadcom portal
• Creating and managing Broadcom support cases