vCenter Server STS service does not start after reboot

Products

• VMware vCenter Server

Issue/Introduction

• After vCenter Server reboot or services restart, STS service does not start.
• Other services may not start
•  /var/log/vmware/vmon/vmon.log log file contains lines similat to

<timestamp>  Wa(03)  host-xxx <sts> Service pre-start command's stderr: Traceback (most recent call last):
<timestamp>  Wa(03)+ host-xxx File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 439, in <module>
<timestamp>  Wa(03)+ host-xxx     if is_action_needed_per_reg_value(STS_INSTALLED_REG_VALUE_NAME):
<timestamp>  Wa(03)+ host-xxx   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 247, in is_action_needed_per_reg_value
<timestamp>  Wa(03)+ host-xxx     current_value = _to_int(get_reg_value(VMIDENTITY_REG_KEY, regKey), 0)
<timestamp>  Wa(03)+ host-xxx   File "/usr/lib/vmidentity/install/sts-prestart-script.py", line 272, in get_reg_value
<timestamp>  Wa(03)+ host-xxx     (rc, stdout, stderr) = run_command(command)
<timestamp>  Wa(03)+ host-xxx   File "/usr/lib/vmware/site-packages/cis/utils.py", line 392, in run_command
<timestamp>  Wa(03)+ host-xxx      close_fds=close_fds)
<timestamp>  Wa(03)+ host-xxx  File "/usr/lib/python3.7/subprocess.py", line 800, in __init__
<timestamp>  Wa(03)+ host-xxx     restore_signals, start_new_session)
<timestamp>  Wa(03)+ host-xxx   File "/usr/lib/python3.7/subprocess.py", line 1551, in _execute_child
<timestamp>  Wa(03)+ host-xxx     raise child_exception_type(errno_num, err_msg, err_filename)
<timestamp>  Wa(03)+ host-xxx OSError: [Errno 8] Exec format error: '/opt/likewise/bin/lwregshell

 

• Inspecting the size of "/opt/likewise/bin/lwregshell" , you can see a size of 0

#ls -lah /opt/likewise/bin/lwregshell

-rwxr-xr-x 1 root root 0 May 24 05:58 /opt/likewise/bin/lwregshell

Environment

VMware vCenter Server 7.x

Resolution

• Ensure that you have a vCenter backup or snapshot (Offline snapshots of all vCenters Server in linked mode) 

https://knowledge.broadcom.com/external/article/313886/vmware-vcenter-in-enhanced-linked-mode-p.html

• Backup lwregshell 

#cp /opt/likewise/bin/lwregshell /opt/likewise/bin/lwregshell.bak

• Copy the lwregshell file from a working vCenter Server (same build)  and Replace the one in problematic vCenter Server.
• Modify permissions

#chmod 755 /opt/likewise/bin/lwregshell

• Restart all vCenter services

#service-control --stop --all 
#service-control --start --all