SDDC Manager 5.2.1 UI fails with message "Server failed to start"

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• SDDC UI shows following message after its reboot. 
  
  

  Server failed to start.

  Welcome! Before you can use this SDDC Manager, please ensure that the conversion of your existing environment to VCF is complete. You can refer to the VCF Admin Guide for more details.

  Check the SDDC Manager Ul Server logs for more details.

• sddcManagerServer.log (Path: /var/log/vmware/vcf/sddc-manager-ui-app/) reports ECONNREFUSED 'Connection refused' messages.

  YYYY-MM-DDTHH:MM ERROR [################] [services/errorHandling.js, error-server-init, errorStateHandler:51] Server is in error state; serving up error html {"serverErrorState":{"code":"#######","details":{"properties":{"cause":"[Function: ve_cause]","code":"ECONNREFUSED","jse_cause":"VCFError [VError]: Welcome! Before you can use this SDDC Manager please ensure that the conversion of your existing environment to VCF is complete. You can refer to the VCF Admin Guide for more details.: connect ECONNREFUSED 127.0.0.1:80\n 
at Object.domainsDataNotAvailable (/opt/vmware/vcf/sddc-manager-ui-app/server/src/errors/VCFError.js:104:5)\n
at getInventoryDomains (/opt/vmware/vcf/sddc-manager-ui-app/server/src/services/inventory.js:272:30)\n
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    
at async init (/opt/vmware/vcf/sddc-manager-ui-app/server/src/servers/sddcManagerServer/init.js:49:32)\n
at async initManager (/opt/vmware/vcf/sddc-manager-ui-app/server/src/servers/sddcManagerServer/index.js:70:9) {\n  
YYYY-MM-DDTHH:MM VERBOSE [fded559fc1c340df] [servers/errorStateServer/index.js, error-server-init, null:48] Incoming request for error server {"url":"/ui/"}

Environment

VCF 5.2.1

Cause

A known RACE condition with nginx and the sddc-manager-ui service when an OS level reboot is performed.

Resolution

Issue is resolved in SDDC 5.2.1.1. Update the SDDC manager to 5.2.1.1 or above to resolve the issue. 

Workaround:

1. SSH into the SDDC Manager Appliance with the vcf user and then elevate to root with su.

2. Take a backup of the sddc-manager-ui-app.service configuration file. 

cp -p /etc/systemd/system/sddc-manager-ui-app.service /home/vcf/

Important: Do not copy the backup file in the /etc/systemd/system/ directory. This will cause stability issues in future updates.


3. Edit this file using the vi editor:

vi /etc/systemd/system/sddc-manager-ui-app.service

4. In the [Unit] section of the file, find the entry starting with 'After=' and add nginx.service

    Example,

[Unit]
After=sddc-manager-ui-db.service nginx.service


5. Reload the system configuration

systemctl daemon-reload

6. Reboot the SDDC Manager Appliance and verify all services start as expected. 

reboot -f