VMware HCX - Unable to upgrade HCX Connector or Cloud Manager after reverting a snapshot

Products

• VMware HCX

Issue/Introduction

• This document is created as a reference to identify and remediate a known issue with VMware HCX Connector or Cloud Manager after reverting a VM snapshot, which is the recommended procedure in the unlikely event that an upgrade fails.
• Reverting a VM snapshot of an HCX Connector or Cloud Manager may cause the system to go out of sync with HCX Service. Any attempt to upgrade the systems will hang.
• The exceptions below can be seen frequently in the /common/logs/admin/app.log logs:

<timestamp> UTC [RemotingService_SvcThread-84, Ent: HybridityAdmin, , TxId: d383cb8c-acae-####-####-########02a] INFO c.v.vchs.hybridity.messaging.JobDao- job df037f3c-714c-####-####-########564 not found or no permission to access it
<timestamp> UTC [RemotingService_SvcThread-84, Ent: HybridityAdmin, , TxId: d383cb8c-acae-####-####-########02a] WARN c.v.vchs.hybridity.messaging.Job- Unable to fill job from JSON {}:{"rowCount":0,"rows":[]}
org.codehaus.jettison.json.JSONException: JSONObject["enterprise"] not found.
at org.codehaus.jettison.json.JSONObject.get(JSONObject.java:369)
at org.codehaus.jettison.json.JSONObject.getString(JSONObject.java:496)
at com.vmware.vchs.hybridity.messaging.Job.ctxFromJson(Job.java:677)
at com.vmware.vchs.hybridity.messaging.Job.fillJobFromJSON(Job.java:490)

Environment

VMware HCX

Cause

Reverting an HCX Connector or Cloud Manager to a previous VM state using snapshots may cause some remote jobs to become orphaned, disrupting the communication process with HCX Service.

Resolution

There is no resolution available at the moment.

If the issue is reported open a case with Broadcom Support.: Creating and managing Broadcom support cases.

Additional Information

Impact/Risks:
The HCX Connector or Cloud Manager can't be upgraded to any release after reverting a snapshot.
There is no impact to any other HCX services or functionality.