" unexpected error while upgrading upgrade unit.command download.cp failed on host".

Products

• VMware NSX
• VMware vSphere ESXi

Issue/Introduction

•  During NSX hosts upgrade, it fails and a similar error is displayed on NSX hosts upgrade page.
  
  "unexpected error while upgrading upgrade unit.command download.cp failed on host".


- Logs from problem ESXi host.

> less var/run/log/nsx-syslog.log:

2025-09-02T19:53:37.302Z nsx-sfhc[2109641]: NSX 2109641 - [nsx@6876 comp="nsx-esx" subcomp="nsxsfhc" tid="2109668" level="WARNING"] Command python -B -c "from __future__ import print_function; import os; os.chdir('/tmp/'); import download_lcp; print(download_lcp.fetch('https://10.##.##.14/repository/3.2.3.2.0.23214453/HostComponents/esx70/nsx-lcp-3.2.3.2.0.23214459-esx70.zip','nsx-lcp-3.2.3.2.0.23214459-esx70.zip'))" failed with return-code 1 (Error occurred while executing download_lcp script: <urlopen error [Errno 110] Connection timed out> ).

- Logs from problem manager node :

> less /var/log/upgrade-coordinator/upgrade-coordinator.log:

2025-09-02T19:53:42.164Z  INFO task-executor-0-workitem-HOST-########-32ff-4a31-b926-0#########d9 UpgradeMessagingServiceImpl 6404 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] decoding message, message bus type=com.vmware.nsx.sfhc.ServiceFabricMessage.SfhcMessage expected type=com.vmware.nsx.sfhc.ServiceFabricMessage.SfhcMessage
2025-09-02T19:53:42.164Z  INFO task-executor-0-workitem-HOST-8#######-32ff-4a31-b926-0c########## UpgradeMessagingServiceImpl 6404 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] decoded type=com.vmware.nsx.sfhc.ServiceFabricMessage.SfhcMessage msg=message_type: MT_OPERATION_STATUS
operation_id: "7#######-da51-49e8-86fb-1#########1"
operation_status {
  status: COMMAND_FAILED
  fabric_object_identifier: "7#######-da51-49e8-86fb-1#########1"
  message: "Error occurred while executing download_lcp script: <urlopen error [Errno 110] Connection timed out>\n"
}
return_same_correlationid: true

2025-09-02T19:53:42.164Z ERROR task-executor-0-workitem-HOST-8#######-32ff-4a31-b926-0########7d9 HostSfhcTemplate 6404 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP30132" level="ERROR" subcomp="upgrade-coordinator"] Command DownloadLcp failed on (8#######-32ff-4a31-b926-0c##########,gtnvmwhv003.cloudcitigtn.datacomm)
2025-09-02T19:53:42.164Z ERROR task-executor-0-workitem-HOST-8#######-32ff-4a31-b926-0########7d9 ESXUpgradeSteps 6404 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP30001" level="ERROR" subcomp="upgrade-coordinator"] Failed to upload bundle retry count is 4/5

>  less var/log/syslog:

2025-09-02T19:53:37.282Z gnsx01 NSX 6404 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Response received: client ########-32ff-4a31-b926-0#########d9, application SERVICE_FABRIC_APPLICATION, messageType com.vmware.nsx.sfhc.ServiceFabricMessage.SfhcMessage, trackingId 6#######-7357-4fcd-bcac-#########f97
2025-09-02T19:53:37.282Z gnsx01 NSX 6404 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Received SfhcMessage message_type: MT_OPERATION_STATUS#012operation_id: "4#####44-67bd-43e5-9d90-a############4"#012operation_status {#012  status: COMMAND_FAILED#012  fabric_object_identifier: "7#######-da51-49e8-86fb-1#########1"#012  message: "Error occurred while executing download_lcp script: <urlopen error [Errno 110] Connection timed out>\n"#012}#012return_same_correlationid: true#012

Environment

VMware NSX

 

Cause

• This issue may occur if the currently configured repository IP is not reachable.
• We can validate by performing net-cat operation   nc -zv <nsx manager FQDN or IP> 443

 

Resolution

•  Execute the following command to obtain the IP address of the current orchestrator / repository node

get service install-upgrade

•  Log in to one of the existing NSX Manager nodes using the CLI and execute the command. This action will configure the specified NSX Manager node as the orchestrator / repository node within the cluster

set repository-ip

 

• The Upgrade-Coordinator is moved to 10.###.##.13
• Re-trying the host upgrade from NSX manager 10.###.##.13 UI and this time the "nsx-lcp-bundle" was successfully pushed from IP 10.###.##.13.