Bare metal edges from separate NSX-T deployments getting configured with same VTEP MAC
search cancel

Bare metal edges from separate NSX-T deployments getting configured with same VTEP MAC

book

Article ID: 345804

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

Symptoms:
When 2 bare metal edges are configured on separate NSX-T deployments, then same VTEP MAC gets allocated to these bare metal edges.

Environment

VMware NSX-T Data Center

Cause

Currently for Bare Metal(BM) edges, NSX-T uses fixed range MAC pool for allocating VTEP MAC. When bare metal edges are deployed in different NSX-T instances, then they use same VTEP MAC.

Resolution

In order to resolve this issue the following workaround should be applied.

Separate L2 Domains : To avoid issues in VTEP traffic drop due to conflicting VTEP MAC address on bare metal edges, user need to use different "transport_vlan" to separate L2 domains. For this user need to configure different VTEP "transport_vlan" for bare metal edges from different NSX deployments.

Configure Routing between separate L2 Domains: In order to communicate between different BM edges' with different transport_vlans, user will need to configure routing. For this, user will need to define a new VTEP IP pool in different subnet range for BM edges in different NSX deployments. User will need to configure routing externally between these BM VTEP IPs(which are now in different subnet networks)

Note: Before applying workaround below user will need to take down time window, because this workaround involves down time.


Workaround:

Steps for applying Workaround

Apply below steps for bare metal edges(having same VTEP MAC) in different NSX deployments
 

Step 1: Making a copy of existing BM edge Hostswitch Uplink profile with new transport_vlan

Step a: Use GET https://{manager-ip}/api/v1/host-switch-profiles/<host-switch-profile-id> API to get existing Hostswitch Uplink profile.

Step b: Now Remove below fields from host-switch-profiles API payload which we got in "Step a" and keep other fields in payload as it is.

    "id": "xxxx",
    "display_name": "xxxx",
    "tags": [],
    "_create_time": 1709667150680,
    "_create_user": "admin",
    "_last_modified_time": 1709873051237,
    "_last_modified_user": "admin",
    "_system_owned": false,
    "_protection": "NOT_PROTECTED",
    "_revision": 2

Step c: Add a meaningful new name with "display_name" field in payload which we got in "Step b"

Step d: Change "transport_vlan" in this new host-switch-profiles API payload. This "transport_vlan" should be different from "transport_vlan" of bare metal edge from another NSX deployment.

Step e: Finally fire POST https://{manager-ip}/api/v1/host-switch-profiles API with payload we prepared in "Step d"

Step 2: Create new VTEP IP pool for BM edge node

Configure a new VTEP IP pool for new BM edge, with different subnet network range, as compared to BM edge from another NSX deployment. This new VTEP IP pool can be configured from UI as shown below. We also need to mention new VTEP Gateway IP for this new VTEP IP pool.

 

Step 3: Configure external physical switches VLANs and VTEP routing

  1. Two separate VLANs must communicate through a layer-3 device, like a router.
  2. Thus, for VTEP to VTEP communication across these bare metal edges(in different L2 domains i.e. having different VLANs) in different NSX deployment, user needs to configure routing between them externally. 
  3. For bare metal edges from different NSX deployments, VTEP IPs should be in different subnet network range. For this configuration we will be using VTEP IP pool which we created in Step 2.
  4. Make sure VTEP gateway IPs on external router are reachable.
  5. Make sure physical switches are configured with correct VLAN.

 

Step 4: Consume newly created Hostswitch Uplink profile and VTEP IP Pool in Bare metal edge node

Finally consume this newly created Hostswitch Uplink profile(from Step 1) and VTEP IP Pool(from Step 2) in Bare metal edge node. For this we can edit bare metal edge node from "System->Fabric→Nodes→Edge Transport Nodes" page and select newly created Hostswitch Uplink profile from drop down as shown in below screenshots.

 

Also select new newly created VTEP ip pool for this BM edge

With this step bare metal edge will get updated with new VTEP IP and new Hostswitch uplink profile. Having different "transport_vlan" makes L2 domain of this BM edge different as compared to BM edge from another NSX-deployment. For connectivity between different L2 domains, external VTEP to VTEP routing needs to be configured by user(as mentioned in Step 3). Thus, having different L2 domains will avoid issues in VTEP traffic drop due to conflicting VTEP MAC address on bare metal edges(across different NSX deployments). 

 

Additional Information

Impact/Risks:
This will cause issues in VTEP traffic, as 2 bare metal edges are using same VTEP MAC address.

Powered by Wolken Software