HCX - Unable to upgrade IX Appliance. Error: "JSONObject["########-####-###-####-de188164179e] not found."

Products

• VMware HCX

Issue/Introduction

HCX IX Appliance may not be able to upgrade to the latest build 4.10. 

The below error/exceptions can be seen in HCX UI and app-engine logs:

Location of App Engine log:

HCX Manager: /common/log/admin/app.log

2024-10-06 11:39:52.247 UTC [InterconnectService_SvcThread-6390, SM:servicemesh-########-####-####-####-b2e6ae078bfe, IX:########-####-####-####-f7b1f9963455, J:3061bc83, , TxId: ########-####-####-####-be8c074d9cd4] ERROR c.v.v.h.s.i.ApplianceLifeCycleJob- ApplianceLifeCycleJob failed in state 
CONSOLIDATE_DEPENDENCIES. Error: JSONObject["########-####-####-####-de188164179e] not found.
org.codehaus.jettison.json.JSONException: JSONObject["########-####-####-####-de188164179e] not found.

2024-10-06 11:40:00.082 UTC [InterconnectService_SvcThread-6392, J:f1149930, , TxId: ########-####-####-####-be8c074d9cd4] ERROR c.v.v.h.s.i.InterconnectRedeploy- SM : servicemesh-########-####-####-####-b2e6ae078bfe, Redeploy of appliance has failed: ########-####-####-####-97be1def0cbc, type: HCX-WAN-IX
2024-10-06 11:40:00.082 UTC [FailureDetectionService_EventListener, , , TxId: ] INFO  c.v.v.h.f.FailedJobEventsListener- Received a Failed Job, jobType: InterconnectServiceJobs workflow type: InterconnectRedeploy
2024-10-06 11:40:00.082 UTC [FailureDetectionService_EventListener, , , TxId: ] INFO  c.v.v.h.f.FailedJobEventsListener- jobAndWorkflowTypesMap :44
2024-10-06 11:40:00.091 UTC [InterconnectService_SvcThread-6398, J:d91fab57, , TxId: ########-####-####-####-be8c074d9cd4] ERROR c.v.v.h.s.i.InterconnectUpgradeAppliance- InterconnectRedeploy failed, errorCode:null. stacktrace:null, errorMessage:Redeploy of Interconnect Appliance failed. Interconnect Service 
Workflow InterconnectRedeploy failed. Error: JSONObject["########-####-####-####-de188164179e] not found.
2024-10-06 11:40:00.091 UTC [InterconnectService_SvcThread-6398, J:d91fab57, , TxId: ########-####-####-####-be8c074d9cd4] WARN  c.v.v.h.s.i.InterconnectUpgradeAppliance- InterconnectUpgradeAppliance failed in state VERIFY_RESPONSE. Error: Interconnect Service Workflow InterconnectRedeploy failed. Error: Redeploy
of Interconnect Appliance failed. Interconnect Service Workflow InterconnectRedeploy failed. Error: JSONObject["########-####-####-####-de188164179e] not found.. Retry attempt: 0.
2024-10-06 11:40:00.099 UTC [FailureDetectionService_EventListener, , , TxId: ] INFO  c.v.v.h.f.FailedJobEventsListener- Existing count of current jobType: InterconnectServiceJobs workflowType: InterconnectRedeploy failure: 0
2024-10-06 11:40:00.099 UTC [FailureDetectionService_EventListener, , , TxId: ] INFO  c.v.v.h.f.FailedJobEventsListener- limit count for jobType: InterconnectServiceJobs workflowType: InterconnectRedeployfailure: 1 is less than: 1
2024-10-06 11:40:00.105 UTC [FailureDetectionService_EventListener, , , TxId: ] INFO  c.v.v.h.f.FailedJobEventsListener- Uploading failure jobType: InterconnectServiceJobs workflowType: InterconnectRedeploy failure: 1

Environment

HCX

Cause

The HCX IX Appliance was renamed from original name "########01" to "########01-NEW" in the vCenter Inventory.

Resolution

1. Named IX Appliance back to original name "########01"


2. "RESYNC" the service mesh. (will Resync both source and destination).


3. Retry to Upgrade IX-Appliance.