NSX Manager upgrade on orchestrator node failed on "resume_other_nodes" step 9 due to missing local hostname entry in /etc/hosts.

Products

• VMware NSX Networking

Issue/Introduction

• NSX Manager upgrade fails to start services on the non-orchestrator nodes on step 9 (resume_other_nodes).
• From admin CLI on the orchestrator node run "get upgrade progress-status"
  • Output for node that failed to start services: "Step start_manager on <manager-UUID-redacted> failed"
• Check the cbm.log of the failed NSX Manager node:

/var/log/cbm/cbm.log

2024-09-11T12:38:09.197Z  INFO WrapperSimpleAppMain EntityManager 31590 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="cbm"] Defined a local entity: Entity{entityType=api, entityUuid=<UUID-redacted>, address=/<ip-address>, portNumber=443, certificate=-----BEGIN CERTIFICATE-----
<certificate-redacted>
-----END CERTIFICATE-----
, isInCluster=Optional.empty, fqdn=, networkPrefixLength=22}

In above cbm.log, "fqdn=" field is missing its own FQDN.

Environment

VMware NSX

Cause

Manually editing the /etc/hosts entry to change the IP address/FQDN/hostname on the NSX Manager VMs to anything other than the NSX Manager's info of itself causes issues with the upgrade workflow.

Example:

root@nsx-mgr:~# cat etc/hosts
127.0.0.1       localhost
<incorrect-NSX-Manager-IP-address>      <incorrect-NSX-Manager-FQDN> <incorrect-NSX-Manager-hostname>

Resolution

Manually edit the /etc/hosts file to the correct IP address/FQDN/hostname, save the change and reboot the NSX Manager VM.