When attaching compute-only nodes to VMware vSAN HCI Mesh Configurations Spanning L3 Networks the vSAN vmkernel for Traffic is attaching to the wrong portgroup

Products

• VMware vSAN 8.x

Issue/Introduction

When expanding a compute only client cluster with hosts spanning across L3 networks the hosts' vmkernel interface is getting attached to wrong portgroup.

Environment

VCF Add Host workflow 

Server Cluster Hosts -> Spanned in different L2/L3 VLAN
Client Cluster Hosts -> Spanned in different VLAN from server cluster hosts
------------------------------------------------------------------------------
HCI Mesh environment in SDDC Manager:
1.  Client cluster is compute cluster.
2.  Host being added to the compute client is from different network than existing hosts in the cluster.

Resolution

First the Host configuration must be modified, and then the SDDC Manager database corrected to reflect the change:

 

Modifying the Host Configuration

1. In vCenter, navigate to the vSphere Distributed Switch (VDS)
2. Add a New Portgroup
   1. The name of the portgroup should conform to SDDC Manager Standards
3. Assign the new Portgroup to the vmkernel for vSAN
   1. Networking -> Add and Manage Hosts
   2. When selecting a task, select "Manage host networking"
   3. Select the Host on which we want to modify the portgroup
   4. Select the portgroup we want to assign to the Host
4. The Host should now be configured correctly

 

Updating the SDDC Manager Database

1.   Take a snapshot of the SSDC Manager in case changes must be reverted

2.   Update the VCF inventory for each Domain by executing the below steps in SDDC Manager as a root user:

curl localhost/inventory/extensions/vi/domainInventory > di.json

3.  List the domain names

cat di.json | jq '.[].domain.name'

4.   Identify the domain name (for which we have updated nics) and prepare payload.json. Here DOMAIN-NAME-INDEX starts from 0. This file contains domain inventory information for the specific domain.

cat di.json | jq '.[<DOMAIN-NAME-INDEX>]' > payload.json

                  for example if 

cat di.json | jq '.[].domain.name'
"domain1"
"domain2"
 
And if we want to modify hosts belongs to domain1 we should execute
cat di.json | jq '.[0]' > payload.json
 
for domain2 we should execute 
cat di.json | jq '.[1]' > payload.json

 

5.   Modify the payload JSON - add the new portgroup in the domain inventory payload under vds section

 

6.   Update the SDDC inventory using curl

curl -s localhost/inventory/extensions/vi/clusters -XPOST -H "Content-type: application/json" -d @payload.json

 

7.   To verify if the inventory is updated correctly log into SDDC manager and check the vds table if the new portgroup is added