"Upgrade vibs operation failed to complete within alloted time" during NSX host upgrade

Products

• VMware NSX

Issue/Introduction

• Upgrading ESXi hosts in NSX fails with error: Upgrade vibs operation failed to complete within alloted time.

• Log lines similar to the below are encountered on the NSX Manager in /var/log/upgrade-coordinator/upgrade-coordinator.log

  INFO http-nio-127.0.0.1-7442-exec-4 UpgradeCoordinatorFacadeImpl 486863 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Component: HOST, status: FAILED, % complete: 0.0, details: Upgrade vibs operation failed to complete within alloted time., canSkip: false

• vLCM is enabled on the Transport Node cluster.
• Compliance is not in healthy state for the relevant cluster/hosts in vCenter under Lifecycle Manager.
• Patch download source under Lifecycle Manager / Settings / Patch Setup shows Not Connected with sources using hostupdate.vmware.com

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Cause

Lifecycle Manager cannot download the necessary patches online to remediate compliance to the Host images.
In turn, the NSX upgrade of ESXi hosts fails to complete because of the above condition.

This behavior is documented in the following article: Error: A general system error occurred: Failed to download VIB(s): Error: HTTP Error Code: 403, vLCM fails to download the ESXi patches and images from online repositories

Resolution

After remediating the Lifecycle Manager compliance, retry the NSX upgrade.