Aria Suite Lifecycle UI stuck indefinitely during patch process

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Aria Suite Lifecycle UI fails to load the UI when patching Aria Suite Lifecycle as per KB426230.
• The UI prompts the service initialization screen and is indefinitely stuck on the same.
  
• The /var/log/vrlcm/patchcli.log states patch installation was completed successful.
  

  Restarting server
  <timestamp> patch instructions script completed
  <timestamp> - main - INFO - patch installation process ended.
  <timestamp> - main - INFO - patch installation completed.

• The bootstrap service is seen to be stuck INPROGRESS indefinitely, when evaluating service status using command: vrlcm-cli --health-status
  

  bootstrap ------------- In progress
  postgres ------------- Not Running                  
  vrlcm-server -------------Not Running 
  blackstone ------------- Not Running    

  Note: Alternatively, the 'postgres', 'vrlcm-server', 'blackstone' services may alternatively be listed as 'Running', yet be non-functional.
• The /var/log/vrlcm/vmware-vrlcm.log show 'No space left on device' when starting services or when writing to the log files.
  fileintegrity: Exception in IntegrityChecker, [Errno 28] No space left on device.
• The df -h command shows the /dev/sda4 partition to be at/ nearing 100% utilization. 
  

Environment

• Aria Suite Lifecycle 8.18

Cause

• This issue is caused when the /dev/sda4 partition hits at 100% utilization mid patching, causing failures when attempting to log or load service packages to initialize the services.

Resolution

• To resolve this issue, start by reverting the Aria Suite Lifecycle to the snapshot taken prior initiating the patch. 
  
  • Note: This is recommended as in the event of having to increase disk space, we would be required to clear out any existing snapshots, which is not recommended mid-upgrade / mid-patch. 
• Attempt to identify and isolate any possible options to clear disk space on the Aria Suite Lifecycle appliance as per : KB407428
• If the /dev/sda4 partition has a space availability of less than ~ 3 GB:
  • increase the /dev/sda4 partition size as per: KB415268.
• Once done, take a snapshot of the Aria Suite Lifecycle at this state and proceed to patch the Aria Suite Lifecycle as per Step2 of KB426230.
• Validate that the Patch completes successfully and the UI is accessible as expected.