[INTERNAL] Deploying a new vRealize Suite Lifecycle Manager (Aria Suite Lifecycle) Manager Instance via SDDC Manager fails in VCF 5.0

Products

• VMware Aria Suite
• VMware Cloud Foundation

Issue/Introduction

Symptoms:
Checking the vRSLCM VM Console we see the postgres service failing to start 




Similar errors in domainmanager.log: 
 

[c.v.e.s.c.v.v.d.Vrslcm8Deployer,dm-exec-18]  vRSLCM service still not deployed: Failed to get vRSLCM system info
2023-08-10T22:34:57.875+0000 ERROR [vcf_dm,3db1270fe97d0597,3867] [c.v.e.s.v.v.DeployVrslcmContract,dm-exec-18]  vRSLCM deployment failed!
2023-08-10T22:34:57.876+0000 ERROR [vcf_dm,3db1270fe97d0597,3867] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-18]  
[R06C21] VRSLCM_OVF_DEPLOYMENT_FAILED vRealize Suite Lifecycle Manager OVF deployment failed.
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: vRealize Suite Lifecycle Manager OVF deployment failed.
        at com.vmware.evo.sddc.vrealize.vrslcm.DeployVrslcmContract.execute(DeployVrslcmContract.java:158)
        at com.vmware.evo.sddc.vrealize.vrslcm.DeployVrslcmContract.execute(DeployVrslcmContract.java:40)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor461.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

journalctl -xe command in vRSLCM appliance will have the below errors.

Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: Starting VMware Postgres database server...
Sep 01 01:27:44 vRSLCM.domain.com postgres[9517]: pg_ctl: could not open PID file "/var/vmware/vpostgres/current/pgdata/postmaster.pid": Permission denied
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Control process exited, code=exited status=1
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Failed with result 'exit-code'.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: Failed to start VMware Postgres database server.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Service RestartSec=100ms expired, scheduling restart.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Scheduled restart job, restart counter is at 5.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: Stopped VMware Postgres database server.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Start request repeated too quickly.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: vpostgres.service: Failed with result 'exit-code'.
Sep 01 01:27:44 vRSLCM.domain.com systemd[1]: Failed to start VMware Postgres database server.


Environment

VMware Cloud foundation 5.x
VMware Cloud Foundation 5.0
VMware vRealize Suite Lifecycle Manager 8.10.x

Resolution

1. Login to vRSLCM VM via console
2. Change directory to /storage/db/pgdata and check the permissions are set to a minimum of 700
3. If required, set the permissions manually on the pgdata directory to: 
   • chmod -R 700 pgdata
4. Start the postgres service: 
   • service vlcm-service start

Workaround:

1. Login to the vRSLCM instance via SSH
2. Run the following commands to re-initailize the postgres services for vRSLCM

rm -rf /storage/db/pgdata/
/etc/bootstrap/firstboot.d/02-configure-postgres.sh
/etc/bootstrap/firstboot.d/20-start-services
service vlcm-service start

Additional Information

Note: If issue still persists with deploying it could be do to an SSH issue. Check for firewall rules blocking SSH and test SSH connectivity between vRSLCM and SDDC Manager.