SDDC edge cluster deployment fails with message Unable to create logical tier-1 gateway

Products

• VMware NSX
• VMware SDDC Manager

Issue/Introduction

• Adding Edge cluster to management or workload domains task fails on Create and Configure NSX-T Data Center Tier-1 Gateway (see image below)
• When expanded the error message is  similar to  below:
  • "Unable to create logical tier-1 gateway, Cause: Locale + Tier-1 gateway <T1 Name> creation timed out for intent path /infra/tier-1s/<tier 1 UUID>"
• When checking directly in NSX the Edge nodes that have been created in this workflow show no healthy VTEP tunnels in System > Nodes and the Edge shows as down. If the Edge shows healthy with active tunnels, this KB is not applicable.

• Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

 

Environment

VMware NSX
VMware NSX-T Data Center
VMware SDDC Manager

Cause

As part of the workflow of the Edge cluster deployment a number of other tasks are performed. The Edge cluster is created, a tier 0 created and assigned and then a tier 1 created and assigned. However during this process if the Edge nodes have all tunnels down (TEP to TEP tunnels) then the node will go to a down state and the tier 1 may fail to create with the above error message. This can be validated by checking if the Edges being deployed in the workflow have all tunnels in a down state. 

The Edge nodes at a minimum should create tunnels between themselves (Edge to Edge) and if these tunnels are down, the physical network or Edge configuration needs to be reviewed and investigated.

Resolution

Review KB Troubleshooting TEPs in NSX and the Edge configuration used to validate connectivity between Edge nodes.

One quick method of validating is to move both Edges to the same ESXI host temporarily. If the Edge to Edge tunnels go healthy then it is likely a vlan issue on the physical network underlay between the hosts and needs to be reviewed by your networking team.


Once the Edge has active healthy TEP tunnels the workflow can be retried in SDDC and should continue. If the config used is incorrect, changes cannot be made directly to NSX and the workflow must time out in SDDC, this will rollback and can be retried with any necessary changes made on the next deploy attempt.