"No transport zone with overlay type found in transport node" error when running pks-nsx-t-osb-proxy job in Tanzu Kubernetes Grid Integrated Edition

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

Running the pks-nsx-t-osb-proxy job fails with the error "No transport zone with overlay type found in transport node" in Tanzu Kubernetes Grid Integrated Edition (TKGI).

The following are example logs for when this issue occurs:

2021/05/09 13:50:22 Error initializing a NSX-T client: Error getting network manager for cluster No transport zone with overlay type found in transport node zzzz4484-8dac-11eb-9882-0c42a1ca2222
time="2021-05-09T13:50:22Z" level=error msg="Failed to extract transport zone ID from edge cluster xxxxb8ba-e837-4dd2-8f31-f06855951111" pks-networking=networkManager
time="2021-05-09T13:50:22Z" level=error msg="Failed to get NSX provisioning properties: No transport zone with overlay type found in transport node zzzz4484-8dac-11eb-9882-0c42a1ca2222" pks-networking=networkManager

Each Edge Transport Node's switch names must be the same as the switch name of the corresponding Transport Zone. The most likely cause of the above error is that the switch names are not the same between an Edge Transport Node switch and the switch in the corresponding Transport Zone.

Environment

Product Version: 1.10

Resolution

Confirm that there is no Transport Zone endpoints information found for the Transport Node that was referenced in the error:

1. bosh ssh into the PKS API VM.

2. Go to the pks-nsx-t-osb-proxy/config directory.

$ cd /var/vcap/jobs/pks-nsx-t-osb-proxy/config

3. Set a variable with the NSX Manager hostname.

$ NSXMGR=$(grep manager_host config.json | awk '{print $2}' | awk -F'"' '{print $2}')

4. Set a variable with the Transport Node ID from the error logs.

$ TNID=zzzz4484-8dac-11eb-9882-0c42a1ca2222

5. Set the variables for your NSX Manager admin login name and password.

$ NSXUSER=admin
$ NSXPASS='###############'

6. Run the following curl command to get more information about the transport node.

$ curl -v -u "$NSXUSER:$NSXPASS" --cacert nsx_t_ca_cert.pem "https://$NSXMGR/api/v1/transport-nodes/$TNID"

7. Review the JSON output of the curl command. You would find "transport_zone_endpoints" info in two places, (1) inside "host_switch_spec" and (2) at the top-level.

If the "transport_zone_endpoints" info at the top-level is blank, then the switch names between the Transport Zones and Edge Transport Nodes are most likely not the same and they need to be sorted out to match in NSX Manager.

For example:

  "transport_zone_endpoints" : [ ],

Run the following checks to confirm the switch names between the Edge Transport Nodes and Transport Zones.

1. Identify the switch name used by the Edge Transport Nodes:

• Go to NSX Manager UI > System > Fabric > Nodes > Edge Transport Nodes.
• For each Edge Transport Node, click Edit
• For each Switch, note down the Edge Switch Name and Transport Zone(s).

2. Identify the switch name used by the Transport Zones:

• If you are using NSX-T v2.5.x:
  • Go to NSX Manager UI > System > Fabric > Transport Zones.
  • For each Transport Zone, note down the N-VDS Name. This name should match the corresponding switch name in the Edge Transport Node
• If you are using NSX-T v3.x, run the following curl command to get the switch name info from all Transport Zones.  From the JSON output, identify the "host_switch_name" from each Transport Zone. This name should match the corresponding switch name in the Edge Transport Node.

  $ curl -v -u "$NSXUSER:$NSXPASS" --cacert nsx_t_ca_cert.pem "https://$NSXMGR/api/v1/transport-zones"

If any of the corresponding switch names don't match, then that is most likely the cause of the error. These particular switch names have to be changed or sorted out to match in NSX Manager to address the issue.