HCX - Failed to resync service mesh: Error: Failed to realize service instance endpoints. Errors: No configuration found for UUID.

Products

• VMware HCX

Issue/Introduction

NSX UUID mismatch causes a Service Mesh modification to fail.

Error: Service Mesh modification failed. Process Service Mesh failed. Interconnect Service Workflow ConfigureServiceInsertionForAppliance failed. Error: Failed to realize service instance endpoints. Errors: No configuration found for ########-####-####-####-05ec4a772b5c, Failed to realize service instance endpoints. Errors: No configuration found for ########-####-####-####-05ec4a772b5c

 

app.log:
2024-09-17 14:43:35.064 UTC [InterconnectService_SvcThread-2694, IX:########-####-####-####-12a937980777, J:9ef167f7, , TxId: ########-####-####-####-2a3836c22ede] ERROR c.v.v.h.s.i.ConfigureServiceInsertionForAppliance- Failed to realize service instance endpoints. {"success":false,"completed":true,"time":1726584215054,"version":"1.0","errors":[{"code":"E_INVALID_INPUT","message":"No configuration found for ########-####-####-####-05ec4a772b5c"}],"data":{"items":[{"siEndpointId":"HCX-SEP-DIRECT_CONNECT-169.254.###.#_##","extNetworkType":"DIRECT_CONNECT"}]}}.

2024-09-17 14:43:35.065 UTC [InterconnectService_SvcThread-2694, IX:########-####-####-####-12a937980777, J:9ef167f7, , TxId: ########-####-####-####-2a3836c22ede] ERROR c.v.v.h.s.i.ConfigureServiceInsertionForAppliance- ConfigureServiceInsertionForAppliance failed in state PROCESS_SEP_INTENT_RESULT. Error: Failed to realize service instance endpoints. Errors: No configuration found for ########-####-####-####-05ec4a772b5c.
java.lang.RuntimeException: Failed to realize service instance endpoints. Errors: No configuration found for ########-####-####-####-05ec4a772b5c.

Environment

HCX 4.8.2

Cause

HCX utilizes a ServiceInsertion config with VMConAWS deployments which keeps track of NSX-MGR UUID. This UUID may change during NSX Manager re-election process for a 3 node cluster and/or during a password rotation.

Resolution

If you experience the above behavior, open a Service Request with VMware by Broadcom Global Support Services and provide the below information. 

- Collect affected HCX-MGR support bundle with DB dump.
- Have there been any power outages or storage issues in the environment?
- When was the issue first experienced? Detail what actions were being performed.

Alternatively, If this is a new deployment with no active Migrations or Network Extensions - Redeploying the affected HCX-MGR will resolve this behavior.