Deploying a new vRealize Suite Lifecycle Manager (Aria Suite Lifecycle) Manager Instance via SDDC Manager fails in VCF 5.0

Products

• VCF Operations/Automation (formerly VMware Aria Suite)
• VMware Cloud Foundation

Issue/Introduction

Checking the vRSLCM VM Console you see the postgres service failing to start 



Similar errors in domainmanager.log: 

[c.v.e.s.c.v.v.d.Vrslcm8Deployer,dm-exec-18]  vRSLCM service still not deployed: Failed to get vRSLCM system info
YYYY-MM-DDTHH:MM:SS.sssZ ERROR [vcf_dm,3db1270fe97d0597,3867] [c.v.e.s.v.v.DeployVrslcmContract,dm-exec-18]  vRSLCM deployment failed!
YYYY-MM-DDTHH:MM:SS.sssZ ERROR [vcf_dm,3db1270fe97d0597,3867] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-18]  
[R06C21] VRSLCM_OVF_DEPLOYMENT_FAILED vRealize Suite Lifecycle Manager OVF deployment failed.
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: vRealize Suite Lifecycle Manager OVF deployment failed.
        at com.vmware.evo.sddc.vrealize.vrslcm.DeployVrslcmContract.execute(DeployVrslcmContract.java:158)
        at com.vmware.evo.sddc.vrealize.vrslcm.DeployVrslcmContract.execute(DeployVrslcmContract.java:40)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor461.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

journalctl -xe command in vRSLCM appliance will have the below errors.

Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: Starting VMware Postgres database server...
Mth DD HH:MM:SS vRSLCM.domain.com postgres[9517]: pg_ctl: could not open PID file "/var/vmware/vpostgres/current/pgdata/postmaster.pid": Permission denied
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Control process exited, code=exited status=1
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Failed with result 'exit-code'.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: Failed to start VMware Postgres database server.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Service RestartSec=100ms expired, scheduling restart.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Scheduled restart job, restart counter is at 5.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: Stopped VMware Postgres database server.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Start request repeated too quickly.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: vpostgres.service: Failed with result 'exit-code'.
Mth DD HH:MM:SS vRSLCM.domain.com systemd[1]: Failed to start VMware Postgres database server.

Environment

• VMware Cloud foundation 5.x
• VMware Cloud Foundation 5.0
• VMware vRealize Suite Lifecycle Manager 8.10.x

Resolution

1. Login to vRSLCM VM via console
2. Change directory to /storage/db/pgdata and check the permissions are set to a minimum of 700
3. If required, set the permissions manually on the pgdata directory to: 
   • chmod -R 700 pgdata
4. Start the postgres service: 
   • service vlcm-service start

Workaround:

1. Login to the vRSLCM instance via SSH
2. Run the following commands to re-initialize the postgres services for vRSLCM

rm -rf /storage/db/pgdata/
/etc/bootstrap/firstboot.d/02-configure-postgres.sh
/etc/bootstrap/firstboot.d/20-start-services
service vlcm-service start

Additional Information

Note: If issue still persists with deploying it could be do to an SSH issue. Check for firewall rules blocking SSH and test SSH connectivity between vRSLCM and SDDC Manager.