Connectivity Issues Registering Supervisor Cluster with TMC Self-Managed (TMC SM)

Products

• VMware Tanzu Mission Control

Issue/Introduction

Attempts to register a Supervisor Cluster with Tanzu Mission Control Self-Managed (TMC SM) fail. Communication from the Supervisor nodes is unable to reach the TMC SM API, preventing the bootstrap process from initiating.

While the TMC SM API remains accessible from other network segments, the Supervisor nodes specifically time out when attempting to reach the Load Balancer VIP. The presumed traffic path is from the Control Plane VM (CPVM) to the Load Balancer; however, the Supervisor remains unable to establish a handshake even when the Load Balancer appears healthy.

Cause

This failure is caused by the Supervisor's networking behavior where traffic destined for a LoadBalancer-type service is intercepted locally via iptables and redirected internally.

Instead of exiting the node to the Load Balancer VIP (##.##.4.46), the Supervisor’s kube-proxy rules perform a Destination Network Address Translation (DNAT). This redirects the traffic directly to the internal NodePort IP and Port of the TMC Service (e.g., ##.##.12.238:30833). 

The network path shifts as follows:

• Presumed Path: CPVM > Load Balancer > VKS Node
• Actual Path: CPVM > eth1 gateway > VKS Node (Internal NodePort)

The issue occurs when a firewall rule blocks traffic originating from the Supervisor Node's management subnet to the TMC Service Node network (##.##.12.0/24) on the specific NodePort range. Because the traffic never actually reaches the Load Balancer, rules applied only to the VIP are insufficient.

The following iptables chains on the Supervisor confirm this internal redirection:

• -A KUBE-SERVICES -d ##.##.4.46/32 ... -j KUBE-EXT-##
• -A KUBE-SVC-## -m statistic --mode random ... -j KUBE-SEP-##
• -A KUBE-SEP-## -p tcp ... -j DNAT --to-destination ##.##.12.238:30833

Resolution

To resolve this issue, update the underlying network firewall policies to allow the traffic path created by the Supervisor's internal networking.

1. Identify the NodePort: Determine the specific NodePort assigned to the TMC SM service (e.g., 30833).
2. Update Firewall Rules: Modify external firewalls or NSX security groups to permit TCP traffic from the Supervisor Node management subnet to the TMC SM Node network (##.##.12.0/24).
3. Validate Port Access: Ensure the required NodePort range (typically 30000-32767) is open between these subnets to facilitate the internal DNAT redirection.
4. Retry Registration: Once the path to the NodePort is unblocked, the Supervisor nodes can complete the internal redirection and successfully reach the TMC SM API.