SDDC Manager password rotation for vRSLCM fails error - Unable to update vCenter password in vRSLCM.
search cancel

SDDC Manager password rotation for vRSLCM fails error - Unable to update vCenter password in vRSLCM.

book

Article ID: 314615

calendar_today

Updated On:

Products

VMware Aria Suite VMware Cloud Foundation

Issue/Introduction

  • SDDC Manager password rotation for vrslcm fails with below error. 
    Unable to update vCenter password in vRSLCM.

     

  • Error in SDDC Manager UI



  • Error in /var/log/vmware/vcf/operationsmanager/operationsmanager.log

    ERROR [vcf_om,e02dbb963005dc7f,b7c1] [c.v.v.p.u.d.VrslcmVcenterPasswordUpdater,om-exec-13] Failed to update sso credentials in vRSLCM
    com.vmware.evo.sddc.common.vrealize.vrlcm.VrlcmException: Failed to update vCenter[{"messageId":"UNKNOWN_SYSTEM_ERROR",
    "message":"Unknown system error.","eventId":"3e83582a-7ee7-44e7-90d6-fd330e470973","retry":true,
    "exceptionMessage":null,"exceptionStackTrace":"java.lang.NullPointerException
    at java.base/java.lang.String.contentEquals(Unknown Source)
    at com.vmware.vrealize.lcm.drivers.commonplugin.task.UpdateVcenterDetailsTask.execute(UpdateVcenterDetailsTask.java:104)
    at com.vmware.vrealize.lcm.automata.core.TaskThread.run(TaskThread.java:63)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)","localizedMessageId":null,"parameters":null,"properties":{}}]
            at com.vmware.evo.sddc.common.vrealize.vrslcm.VrslcmService.updateVcenter(VrslcmService.java:881)
            at com.vmware.evo.sddc.common.vrealize.vrslcm.VrslcmService.updateVcenterCredentials(VrslcmService.java:892)
            at com.vmware.vcf.passwordmanager.update.dependents.VrslcmVcenterPasswordUpdater.update(VrslcmVcenterPasswordUpdater.java:139)
            at com.vmware.vcf.passwordmanager.update.changers.AbstractPasswordChanger.doUpdaters(AbstractPasswordChanger.java:993)
    
    
    ERROR [vcf_om,e02dbb963005dc7f,b7c1] [c.v.v.p.u.c.AbstractPasswordChanger,om-exec-13] Unable to update vCenter password in vRSLCM
    com.vmware.vcf.passwordmanager.exception.PasswordUpdateException: Unable to update vCenter password in vRSLCM
            at com.vmware.vcf.passwordmanager.update.changers.AbstractPasswordChanger.updateDependantError(AbstractPasswordChanger.java:935)
            at com.vmware.vcf.passwordmanager.update.changers.AbstractPasswordChanger.doUpdaters(AbstractPasswordChanger.java:1007)
    
    
    Caused by: com.vmware.vcf.passwordmanager.exception.PasswordUpdateException: Unable to update vCenter password in vRSLCM
            at com.vmware.vcf.passwordmanager.update.dependents.VrslcmVcenterPasswordUpdater.update(VrslcmVcenterPasswordUpdater.java:180)
            at com.vmware.vcf.passwordmanager.update.changers.AbstractPasswordChanger.doUpdaters(AbstractPasswordChanger.java:993)
            ... 9 common frames omitted
    
    DEBUG [vcf_om,e02dbb963005dc7f,b7c1] [c.v.v.p.u.c.AbstractPasswordChanger,om-exec-13] 
    Error Message : Unable to update vCenter password in vRSLCM, Error Token : E3T6ST, Error Cause : {}
  • vRSLCM locker has the below error when attempting to view passwords
    Failed to validate root password. Provide correct root password to authorize this action.

Environment

  • VMware Cloud Foundation 4.x
  • VMware Cloud Foundation 5.x

Cause

This issue occurs when the vRSLCM upgrade is rolled back from version 8.10 to version 8.8, which causes a discrepancy in the inventory version information between the vRSLCM instance and the SDDC Manager.

Resolution

  1. Take Snapshot of SDDC Manager VM
  2. SSH to SDDC Manager with vcf user and su to root
  3. Connect to platform database
    psql -h localhost -U postgres -d platform

    for 5.1.1

    /usr/pgsql/13/bin/psql -h localhost -U postgres -d platform
  4. Check the vrslcm table
    select id,status,version from vrslcm;

    Sample output

    id 										| status 		| version
    f79c67a0-92fb-471e-9a48-172267801431	  ACTIVE		8.8.2-20080494
  5. Update vRSLCM version to the correct version
    update vrslcm set version='8.8.2.3-20080494';



Additional Information

Impact/Risks:

HIGH: Since the resolution involves manual edits to the SDDC Manager database entries, a snapshot of the SDDC Manager VM is highly recommended prior to making any changes.