Symptoms:
- All services are down on the NSX Manager node with 90% upgrade progress.
- Attempting to retry the upgrade fails.
- The NSX-T Manager UI is unavailable.
- In /var/log/syslog.log similar entries to the below will be observed:
2022-01-16T20:28:03.932Z ngnynm-rochm02 NSX 1110 - [nsx@6876 comp="nsx-manager" subcomp="mpa-client" tid="1421" level="INFO"] [UPGRADE] MpaClientHostService Received: Msg type (com.vmware.nsx.upgrade_agent.PrepareUpgradeMsg)corelationId (e46e9fe9-xxxx-xxxx-xxxx-xxxxxxxxxxxx) from APH (d1dfaab2-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
2022-01-16T20:28:03.933Z ngnynm-rochm02 NSX 1110 - [nsx@6876 comp="nsx-manager" subcomp="upgrade-agent" tid="1421" level="INFO"] Got PrepareUpgradeMsg()
2022-01-16T20:28:51.707Z ngnynm-rochm02 NSX 1110 - [nsx@6876 comp="nsx-manager" subcomp="upgrade-agent" tid="1421" level="INFO"] [SendMsg] Sending message (type:com.vmware.nsx.upgrade_agent.PrepareUpgradeResponseMsg, len:523)
2022-01-16T20:28:51.707Z ngnynm-rochm02 NSX 1110 - [nsx@6876 comp="nsx-manager" subcomp="upgrade-agent" tid="1421" level="ERROR" errorCode="MPA50004"] [SendMsg] failed to send message