"no healthy upstream" error when attempting to access the vCenter Server UI.

Products

• VMware vSphere ESXi

Issue/Introduction

• Following error is observed when trying to access the vCenter Server UI "no healthy upstream" 

• The sts service was in a stopped state, verified using the command: service-control --status --all

• Attempted to manually start the stsd service which failed with the error message "An error occurred while starting service '%(0)s"

• The sts service pre-start failed due to Likewise registry errors which was observed in /var/log/vmware/vmon/vmon.log

YYYY-MM-DDTHH:MM:SSZ In(05) host-####### Received start request for sts
YYYY-MM-DDTHH:MM:SSZ In(05) host-####### <sts-prestart> Constructed command: /usr/bin/python /usr/lib/vmidentity/install/sts-prestart-script.py /var/log/vmware/sso/sts-prestart.log
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr: Traceback (most recent call last):
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 540, in <module>
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr:     raise e
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 536, in <module>
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr:     sts_prestart_fsp_normalization()
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 184, in sts_prestart_fsp_normalization
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr:     lwreg_values = run_cmd("/opt/likewise/bin/lwregshell list_values [HKEY_THIS_MACHINE\\Services\\vmdir]".split())
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 418, in run_cmd
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr:     process = subprocess.run(cmd, input=cmd_input, check=True, capture_output=True, universal_newlines=True)
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######   File "/usr/lib/python3.10/subprocess.py", line 526, in run

YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### <sts> Service pre-start command's stderr:     raise CalledProcessError(retcode, process.args,
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-####### subprocess.CalledProcessError: Command '['/opt/likewise/bin/lwregshell', 'list_values', '[HKEY_THIS_MACHINE\\Services\\vmdir]']' returned non-zero exit status 252.
YYYY-MM-DDTHH:MM:SSZ Wa(03)+ host-#######
YYYY-MM-DDTHH:MM:SSZ Er(02) host-####### <sts> Service pre-start command failed with exit code 1
YYYY-MM-DDTHH:MM:SSZ Wa(03) host-####### [ReadSvcSubStartupData] No startup information from sts.

• Unable to read the lwregshell registry using the following command:  /opt/likewise/bin/lwregshell ls "[HKEY_THIS_MACHINE\Services\vmdir]"

lwregshell (error = 40700 - LWREG_ERROR_NO_SUCH_KEY_OR_VALUE)
No such key or valueCause Justification

Environment

VMware vCenter Server 8.x

Cause

stsd service crashed due to stale likewise processes.

Resolution

• To resolve the issue, reboot the vCenter Sever which will clear the stale likewise processes and start the sts service.

Additional Information

"LWREG_ERROR_NO_SUCH_KEY_OR_VALUE", vCenter update fails with error Internal error occurs during execution of update process