Adding a new host fails when scaling out a VSAN cluster with FileService, NSX-T and vLCM enabled

Products

• VMware vCenter Server
• VMware vSAN
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• When scaling out the vSAN cluster (FileService, NSXT-CVDS and vLCM enabled) by adding a new host directly to the cluster, it is expected that the NSX installation can be done automatically, but it fails during the pre-check of cluster.
• You see an error from EAM/FS that FS agent VM cannot deploy when adding host to cluster.

Environment

VMware vCenter Server 7.0.x
VMware vSAN 7.0.x
VMware vSphere ESXi 7.0.x

Cause

The error occurs due to the vSAN Health check failure status, NSX-T expects a healthy vSAN to be configured when adding a new node.

Resolution

To work around the issue:
1. Before scaling out the VSAN cluster(VSAN Fileservice, NSXT-CVDS and vLCM enabled) with new host, verify there are no cluster health check errors.
2. Add a new host in the Datacenter and put in maintenance mode.
3. Add the new host to VDS.
4. Install and configure NSX on the new host.
5. Create vSAN vmknics on the host
6. Exit the host from maintenance mode
7. Add the host into the cluster
8. Create disk group on the new host (EAM will retry the FS agent deployment)

Note:
In step 7:

• The EAM will try to deploy FileService agent, but as no disk group is created on the new host yet, the task will fail and prompt error alert.
• The "Apply NSX Solution"(with vLCM remediation) task will still be triggered and fail due to the EAM error alert, but the NSX component is installed/configured in step 4 already

In step 8:

• After disk group is created, wait for a while, "Remediate vSAN file service" will be triggered and the EAM will retry the FileService agent deployment successfully. The cluster with the new host goes back to healthy status and the VMs starts to migrate to this new host.