vMotion is failing to upgraded ESXi host with error mentioning that "nsxa down"

Products

• VMware NSX
• VMware vSphere ESXi

Issue/Introduction

• After NSX update on ESXi host, vMotion to this host is failing with the following message:

  "Currently connected network interface" 'Network adapter X' uses network 'DVSwitch[50 3e ## ## ## ## ## ##-## ## ## ## ## ## 42 4e] NSX port group [dvportgroup-#####](nsxa down)', which is not accessible."

• Moving the affected host out of the cluster is not removing NSX configuration from the ESXi host.
• Trying to remove NSX from ESXi host with "nsx del" over nsxcli is not completing.
• Adding the host back to the NSX cluster, NSX installation on the host fails with generic error.
• Restarting NSX services (nsx-opsagent, nsx-cfgagent or nsx-nestdb) on ESXi host or rebooting ESXi host does not resolve the issue.
• The following messages continuously being reports in the affected ESXi host /var/log/hostd.log file:

   info hostd[2100639] [Originator@6876 sub=Solo.Vmomi opID=nsxDaVim-e3-b9b9 user=dcui] Activation finished; <<########-####-####-####-############, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 41028'>>, ha-host-access-manager, vim.host.HostAccessManager.u
  pdateLockdownExceptions>
   verbose hostd[2100639] [Originator@6876 sub=Solo.Vmomi opID=nsxDaVim-e3-b9b9 user=dcui] Arg users:
  --> (string) [
  -->    "DOMAIN\username1",
  -->    "DOMAIN\username2",
  -->    "mux_user",
  -->    "nsx-user",
  -->    "root",
  -->    "da-user"
  --> ]
   info hostd[2100639] [Originator@6876 sub=Solo.Vmomi opID=nsxDaVim-e3-b9b9 user=dcui] Throw vim.fault.UserNotFound
   info hostd[2100639] [Originator@6876 sub=Solo.Vmomi opID=nsxDaVim-e3-b9b9 user=dcui] Result:
  --> (vim.fault.UserNotFound) {
  -->    principal = "DOMAIN\username1",
  -->    unresolved = false,
  -->    msg = "",
  --> }

• A user that is added in Lockdown Mode Exception User list no longer exists in the system.

Environment

VMware NSX

VMware NSX-T Data Center

VMware vSphere ESXi

Cause

The issue is caused because the user that is added to Lockdown Mode Exception User list in ESXi host configuration no longer exists in the system. ESXi host keeps trying to verify the user list but it cannot progress further because one of the Exception users are no longer in the system.

Resolution

Remove the user from Lockdown Mode Exception User list in ESXi which no longer exists:

1. Click on the affected ESXi host in the vCenter UI
2. Click on Configuration
3. Click on Security Profile
4. Next to Lockdown Mode click on EDIT....
5. One new window pops up as per below screenshot click on Exception Users and remove the non existing user from the list.
   
6. If you are using Host Profiles, please make sure to remove the non existing user from the Host Profile configuration.

Additional Information

vMotion is blocked due to error "Currently connected network interface ... uses network (nsxa down), which is not accessible"
Enabling or disabling Lockdown mode on an ESXi host