VCF Cluster expansion fails during host add with non-standard PNICs

Products

• VMware Cloud Foundation

Issue/Introduction

Instructions to utilize API calls to expand a cluster with non-standard VMNIC configurations.

Symptoms:
The host add to expand a cluster showing the following error in the domain.log:

CLUSTER_HOST_VMNIC_CONFIGURATION_NON_HOMOGENEOUS All the hosts in the cluster must have homogeneous vmnic configuration.

This failure occurs because the hosts in the cluster are not using VMNIC0 and VMNIC1 (standard VCF VMNICs) for the configuration.
 

Environment

VMware Cloud Foundation 3.10.x
VMware Cloud Foundation 4.x

Cause

Environment is not utilizing VMNIC0 and VMNIC1 for the host connectivity. 

Resolution

Utilize the API to configure hosts utilizing the non-standard VMNICs.  In the following example VMNIC1 and VMNIC2 will be utilized.  These examples are utilizing the developer center.

1. Commission the hosts following the standard UI process.
2. Gather the UUID for the cluster being expanded
   • Developer Center > Api Explorer > APIs for managing clusters > GET
   • Click execute
3. Go to the PATCH option in the APIs for managing clusters
   • There is an option to auto fill with an example of flags for expanding the cluster
4. Enter the cluster ID 
5. For the VMNIC reconfiguration, here is an example:

{
  "clusterExpansionSpec": {
    "hostSpecs": [ {
      "id": "30e2a984-25af-4c2c-aafe-1700b433431f",
      "hostNetworkSpec": {
        "vmNics": [ {
          "id": "vmnic1",
          "vdsName": "sfo-m01-c101-vds01"
        },{
          "id": "vmnic2",
          "vdsName": "sfo-m01-c101-vds01"
        } ]
      }
    }

  ],
  "interRackExpansion":false
  }
}

5. Click execute
6. Cluster should expand with the host.  If the vSAN configuration is not as expected, that can be reconfigured after expansion.

Additional Information

Official documentation for this process:
https://docs.vmware.com/en/VMware-Validated-Design/services/sddc-multiple-pnic-configuration/GUID-F0BB3BA3-25B3-4FF0-B961-2C8293913D57.html