NSX Upgrade Stuck at Begin Upgrade Progress

Products

• VMware NSX Networking

Issue/Introduction

• NSX Upgrade Stuck at Begin Upgrade Progress
• Error: Unable to connect to File /repository/4.1.2.4.0.23786733/UC/upgrade_backup.py
• /var/log/proton/nsxapi.log in the NSX manager node report error similar to below
  
  2024-06-21T05:22:54.063Z  INFO http-nio-127.0.0.1-7440-exec-8 RepositoryServiceImpl 1968120 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" reqId="674c####-e###-4####-8####-0####c1d#####" subcomp="manager" username="admin"] fqdn_required flag for (cfb####-f###-e###-a###-c0#####ec4 : 10.xx.xxx.xx) node is true.
2024-06-21T05:22:54.063Z  INFO http-nio-127.0.0.1-7440-exec-8 Utils 1968120 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="674c####-e###-4####-8####-0####c1d#####" subcomp="manager" username="admin"] getFqdnFromIp(): invoked with Ip Address 10.xx.xxx.xx
2024-06-21T05:22:54.242Z  INFO http-nio-127.0.0.1-7440-exec-8 IpAddressUtils 1968120 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="674c####-e###-4####-8####-0####c1d#####" subcomp="manager" username="admin"] getFqdnFromIp(): returning FQDN : 10.xx.xxx.xx for IP: 10.xx.xxx.xx
2024-06-21T05:22:54.242Z  INFO http-nio-127.0.0.1-7440-exec-8 DnsLookupProviderImpl 1968120 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="674c####-e###-4####-8####-0####c1d#####" subcomp="manager" username="admin"] DNS Resolve: Reverse DNS lookup for IP address 10.xx.xxx.xx resolved FQDN 10.xx.xxx.xx
2024-06-21T05:22:54.277Z  INFO RepoSyncThread-1718947374276 RepoSyncServiceImpl 1968120 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Starting Repo sync thread RepoSyncThread-1718947374276
2024-06-21T05:22:57.502Z  INFO RepoSyncThread-1718947374276 RepoSyncFileHelper 1968120 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Command to get server info for https://10.xx.xxx.xx:443/repository/4.1.2.4.0.23786733/Manager/vmware-mount/libcrypto.so.0.9.8 returned result CommandResultImpl [commandName=null, pid=2748837, status=FAILED, errorCode=51, errorMessage=curl_wrapper: (51) SSL: no alternative certificate subject name matches target host name '10.xx.xxx.xx'

 

• The error look similar to the screenshot below
  

Environment

NSX 4.1.2.1

Cause

DNS resolution is not working for the NSX manager nodes

Resolution

To resolve the issue, make sure DNS entries are made for the NSX manager nodes and the forward/reverse DNS lookups work fine for all the 3 manager nodes.