vRSLCM Inventory sync for vRLI fails with error: Failed to execute vRLI insertion operation on SDDC Manager

Products

• VMware Aria Suite
• VMware Cloud Foundation

Issue/Introduction

Symptoms:

vRSLCM Inventory sync for vRLI fails with error: Failed to execute vRLI insertion operation on SDDC Manager

SDDC Manager operationsmanager.log which shows the credential query task is initiated

2024-01-15T19:31:34.541+0000 DEBUG [vcf_om,4224d92c29f4e49b,5667] [c.v.v.p.s.PasswordLookupService,http-nio-127.0.0.1-7300-exec-3] Fetch credentials from Entity type VRLI

2024-01-15T19:31:34.541+0000 DEBUG [vcf_om,4224d92c29f4e49b,807e] [c.v.v.p.helper.CredentialHelper,om-exec-15] Query credentials based on entityType: VRLI, SkipPasswords true

2024-01-15T19:31:34.545+0000 DEBUG [vcf_om,4224d92c29f4e49b,807e] [c.v.v.p.helper.CredentialHelper,om-exec-15] Size from Credentials query API from page 0 : 0

2024-01-15T19:31:34.545+0000 INFO [vcf_om,4224d92c29f4e49b,9e78] [c.v.v.p.l.t.LoginsightFetcherTask,om-exec-6] Processing vrlis, vrli primary node:<vRLI Node FQDN>

2024-01-15T19:31:34.545+0000 INFO [vcf_om,4224d92c29f4e49b,9e78] [c.v.v.p.l.t.LoginsightFetcherTask,om-exec-6] Processing worker node <vRLI Node FQDN of vRLI VIP>

2024-01-15T19:31:34.545+0000 INFO [vcf_om,4224d92c29f4e49b,9e78] [c.v.v.p.l.t.LoginsightFetcherTask,om-exec-6] Processing worker node <vRLI Node FQDN of vRLI VIP>

2024-01-15T19:31:34.545+0000 ERROR [vcf_om,4224d92c29f4e49b,5667] [c.v.v.p.l.u.LookupResponseProcessor,http-nio-127.0.0.1-7300-exec-3] Skipping/Couldn't fetch credentials for entityType : VRLI

 

Error in SDDC Manager domainmanager.log

2024-01-15T20:27:53.631+0000 ERROR [vcf_dm,66511663d9d351b3,f852] [c.v.e.s.c.u.c.SshCommandExecuter,http-nio-127.0.0.1-7200-exec-6] Could not connect to the SSH server @<vRLI Node FQDN> for configuration.

com.jcraft.jsch.JSchException: Algorithm negotiation fail

 

2024-01-15T20:27:53.632+0000 ERROR [vcf_dm,66511663d9d351b3,f852] [c.v.v.v.c.impl.VrliManagerImpl,http-nio-127.0.0.1-7200-exec-6] Failed to insert vRLI

com.vmware.evo.sddc.common.util.command.CommandExecuterException: SSH: Failed to establish SSH session to <vRLI Node FQDN>

2024-01-15T20:27:53.635+0000 ERROR [vcf_dm,66511663d9d351b3,f852] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-6] [TDMHQ4] PRODUCT_REGISTRATION_FAILED Failed to register 578bbf0f-da24-4049-xxxx-xxxxxxxxxxxx:vrli in the SDDC Manager.

com.vmware.evo.sddc.common.core.error.InvalidStateException: Failed to register 578bbf0f-da24-4049-xxxx-xxxxxxxxxxxx:vrli in the SDDC Manager.

Caused by: com.vmware.evo.sddc.common.core.error.InvalidStateException: Failed to execute vRLI insertion operation on SDDC Manager.

        at com.vmware.vcf.vrealize.controller.impl.VrliManagerImpl.insertVrli(VrliManagerImpl.java:171)

        at com.vmware.vcf.vrealize.controller.v1.VrliPublicController.insertVrli(VrliPublicController.java:166)

        ... 117 common frames omitted

Caused by: com.vmware.evo.sddc.common.util.command.CommandExecuterException: SSH: Failed to establish SSH session to <vRLI Node FQDN>

Cause

KB Steps for Removing SHA1 weak Algorithms/Ciphers from all VMware Aria Products implemented on the environment which resulted in mismatched ssh algorithms on the server (vRLI) and the client side (VCF)

Resolution

1. Take snapshot of the vRLI VMs
2. SSH to vRLI nodes with root credentials
3. Edit the sshd_config file
   • vi /etc/ssh/sshd_config
4. Add the below parameters in the Ciphers config
   • aes256-ctr,aes192-ctr,aes128-ctr
5. Save the changes to /etc/ssh/sshd_config
   • ESC
   • wq!
6. Restart the SSH service
   • systemctl restart sshd
7. Retry the inventory sync from vRSLCM

 

Additional Information

Impact/Risks:

vRSLCM inventory sync for vRLI will fail and SDDC will have no record of the vRLI instance