Adding hosts to cluster fails in SDDC Manager with UNABLE_TO_CREATE_TRANSPORT_NODE

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

The SDDC Manager /var/log/vmware/vcf/domainmanager/domainmanager.log contains the following errors:

ERROR [vcf_dm,69cbd8f9377b09513de03867b9b92104,9d3a] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-10]  [F8HFAV] UNABLE_TO_CREATE_TRANSPORT_NODE_COLLECTION Unable to create transport node collection with profile ######-affc-xxxx-95b7-####### on compute collection xxxx-f4d2-48a9-###-xxxxxxxx:domain-c1x through nsxmanager <FQDN>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Unable to create transport node collection with profile ####-affc-xxxx-95b7-####### on compute collection xxxx-f4d2-48a9-###-xxxxxxxx:domain-c1x through nsxmanager <FQDN> ... Caused by: java.lang.RuntimeException: Failed to realize transport node. Please refer logs. 

The NSX Manager /var/log/proton/policy-ui.log contains the underlying error identifying the MTU rejection:

Http failure response for https://<FQDN>/policy/api/v1/infra/sites/default

Additionally, even after the MTU configuration is corrected in the Uplink Profile, the Transport Node Collection (TNC) state remains indefinitely stuck in FAILED_TO_REALIZE within NSX, and subsequent SDDC Manager operations continue to fail.

Environment

VMware Cloud Foundation 5.2.x
VMware Cloud Foundation 9.x

VMware NSX 4.2.x
VMware NSX 9.0.x, 9.1.x

Cause

This issue manifests in two phases:

Initial Failure (MTU Configuration): An NSX Uplink Profile mapped to the Transport Node Profile (TNP) contains an explicit MTU value (e.g., 9000). NSX natively rejects this configuration (Error 9531) because MTU settings on a vSphere Distributed Switch (VDS) are managed exclusively by vCenter Server.
Persistent State Failure: In NSX 4.2.x, the state of a Transport Node Collection is calculated dynamically. If the underlying installation issue is not resolved within 45 minutes, the TNC state becomes cached as FAILED_TO_REALIZE. Simply removing the MTU value from the Uplink Profile does not automatically trigger the TNC State API to recalculate the cluster's state, leaving the TNC in a failed status despite individual Transport Nodes reporting SUCCESS

Resolution

WORKAROUND

To restore functionality, the MTU value must be removed from the Uplink Profile, and the TNC state must be explicitly recalculated.

Step 1: Remove MTU from the Uplink Profile

UI Method:

Log in to the NSX Manager UI.
Navigate to System > Fabric > Profiles > Uplink Profiles.
Edit the affected Uplink Profile.
Clear the MTU field entirely and save the configuration.

API Method:

Execute a GET request to retrieve the profile payload: GET /policy/api/v1/infra/host-switch-profiles/<host-switch-profile-id>
Remove the "mtu" key-value pair from the JSON response.
Execute a PUT request with the modified payload to update the profile: PUT /policy/api/v1/infra/host-switch-profiles/<host-switch-profile-id>

Step 2: Recalculate Transport Node Collection State

Force NSX to recalculate the TNC state using one of the following methods:

UI Method: In the NSX Manager UI, navigate to System > Fabric > Nodes > Host Transport Nodes. Loading this page automatically invokes the state API and recalculates the latest TNC state. If any nodes are not in SUCCESS status then reconfigure NSX on those
API Method: Execute the following GET request against the NSX Manager API: GET /policy/api/v1/infra/sites/<site-id>/enforcement-points/<enforcementpoint-id>/transport-node-collections/<tnc-id>/state

Step 3: Retry SDDC Manager Workflow Return to the SDDC Manager UI and retry the "Add Host" operation.

Additional Information

https://developer.broadcom.com/xapis/nsx-t-data-center-rest-api/latest/all_types.html
https://developer.broadcom.com/xapis/nsx-t-data-center-rest-api/latest/schemas_TransportNodeCollectionState.html

The Transport Node Collection state calculation logic will be resolved in future release of NSX.

No maintenance window is required to apply the workaround.