Inventory sync fails with error: LCMVCF19116: Status code for response obtained from SDDC Manager is not in valid range.

Products

• VMware Cloud Foundation

Issue/Introduction

 

Inventory sync fails with error:

LCMVCF19116: Status code for response obtained from SDDC Manager is not in valid range.

 

/var/log/vmware/vcf/domainmanager/domainmanager.log shows:

2023-05-11T16:15:32.333-0400 DEBUG [vcf_dm,ef24a6db1af44988,1a83] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-10] Processing localizable exception Failed to register 11111-222-333-4444-55555555:vrli in the SDDC Manager.

2023-05-11T16:15:32.335-0400 ERROR [vcf_dm,ef24a6db1af44988,1a83] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-10] [CCHIN] PRODUCT_REGISTRATION_FAILED Failed to register 11111-222-333-4444-55555555:vrli in the SDDC Manager.
com.vmware.evo.sddc.common.core.error.InvalidStateException: Failed to register 11111-222-333-4444-55555555:vrli in the SDDC Manager.
       at com.vmware.vcf.vrealize.controller.v1.VrliPublicController.insertVrli(VrliPublicController.java:170)
       at com.vmware.vcf.vrealize.controller.v1.VrliPublicController$$FastClassBySpringCGLIB$$b0818a4b.invoke(<generated>)

Caused by: com.vmware.evo.sddc.common.core.error.InvalidStateException: Failed to execute vRLI insertion operation on SDDC Manager.
       at com.vmware.vcf.vrealize.controller.impl.VrliManagerImpl.insertVrli(VrliManagerImpl.java:171)
       at com.vmware.vcf.vrealize.controller.v1.VrliPublicController.insertVrli(VrliPublicController.java:166)
       ... 116 common frames omitted

Caused by: com.vmware.evo.sddc.common.core.error.InvalidStateException: vRealize Lifecycle Manager Suite hostname.example.com is not available
       at com.vmware.vcf.vrealize.controller.impl.VrealizeRestAdapter.getVrliExcludedComponents(VrealizeRestAdapter.java:443)
       at com.vmware.vcf.vrealize.controller.impl.VrliManagerImpl.connectWithDomain(VrliManagerImpl.java:257)
       at com.vmware.vcf.vrealize.controller.impl.VrliManagerImpl.insertVrli(VrliManagerImpl.java:166)
       ... 117 common frames omitted

2023-05-11T16:15:32.341-0400 DEBUG [vcf_dm,ef24a6db1af44988,1a83] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-10] 
Handler Error Response: {"errorCode":"PRODUCT_REGISTRATION_FAILED","arguments":["11111-222-333-4444-55555555:vrli"],
"message":"Failed to register 11111-222-333-4444-55555555:vrli in the SDDC Manager.",
"causes":[{"type":"com.vmware.evo.sddc.common.core.error.InvalidStateException","message":"Failed to execute vRLI insertion operation on SDDC Manager."},
{"type":"com.vmware.evo.sddc.common.core.error.InvalidStateException",
"message":"vRealize Lifecycle Manager Suite hostname.example.com is not available"}],"referenceToken":"CCHIN"}

Environment

VMware Cloud Foundation 4.x

Cause

This is due to the vRealize Suite Lifecycle Manager currently being in an errored state within SDDC Manager.

We can review this status in the SDDC Manager UI in the vRealize tab:

 

Resolution

Update vRSLCM status to ACTIVE in SDDC Manager DB using the following steps:

1. Take a snapshot/backup of SDDC Manager before making changes. 
2. SSH into the SDDC Manager using the vcf user and then su to root.
3. Connect to the SDDC Manager platform DB: psql -h localhost -U postgres -d platform
4. Verify the current status of vRSLCM from the vrslcm table: select * from vrslcm;
5. Change vrslcm state to ACTIVE: update vrslcm set status = 'ACTIVE' ;
6. Retry Inventory Sync from the vRSLCM UI.
7. If the Inventory Sync fails again, or if the vRSLCM status within SDDC Manager goes back to ERROR, additional investigation is required in both the SDDC Manager logs and the vRSLCM logs to determine why the status is being set to ERROR.

Additional Information

Impact/Risks:

 

 

HIGH: Since the resolution involves manual edits to the SDDC Manager database entries, a snapshot of the SDDC Manager VM is highly recommended prior to making any changes.