HCX Manager upgrade failing "Error fetching public key from file"

Products

• VMware HCX

Issue/Introduction

• HCX Manager upgrade from the hybridity page (port 443) System -> Administration -> System Updates fails with the following error:
  
  "Error fetching public key from file, cannot proceed to validate the signed bundle"
  
  
  
  

 

• On the HCX Admin page (accessible via https://<hcx-manager-ip-fqdn>:9443), when navigating to Administration > Upgrade > Upgrade > Choose File and attempting the upgrade, it fails with the same error as mentioned above.

 

• Below errors are noticed in HCX manager appliance-management.log under /common/logs/appliance-management/

2024-07-26 05:35:59.280 UTC [https-jsse-nio-9443-exec-8, , ] WARN  c.g.jsonrpc4j.JsonRpcBasicServer- Error in JSON-RPC Service
java.lang.RuntimeException: Error fetching key from file , cannot proceed to validate the signed bundle

 

• The correct download files are present under /common/tmp/upgrade/download
  
  

[/common/tmp/upgrade/download ]$ ls -ltrh
total 4.1G
-rw-r--r-- 1 admin root 4.1G Feb  4 19:30 VMware-HCX-Cloud-upgrade-bundle-4.8.2.0-23247145.tar.gz
-rw-r--r-- 1 admin root 2.0K Feb  4 19:34 VMware-HCX-Cloud-upgrade-bundle-4.8.2.0-23247145.tar.gz.sha256.sig

• On HCX Manager, the files listed under the directory /common/ks, show that the specific file ovf_klnext.public is of 0 KB. 

Environment

VMware HCX

Cause

Issue is due to the file "ovf_klnext.public" being corrupted or invalid.

Resolution

• Take a backup from HCX Admin page(port 9443) and create a snapshot of the HCX Manager VM from vCenter.
• Copy the file ovf_klnext.public from a working HCX Manager (of the same HCX version) to the /common/ks/ directory of the affected HCX Manager.
• On the HCX Admin Page at port 9443 (accessible via https://<hcx-manager-ip-fqdn>:9443), navigate to Administration > Upgrade > Upgrade > Choose File > Select the HCX Upgrade bundle and then retry the upgrade.

Additional Information

HCX upgrade bundle can be downloaded from the Broadcom Support Portal. Next to your profile on top right, click the dropdown and go to VMware Cloud Foundation -> My Downloads -> VMware HCX
Reference: Download Broadcom products and software