Lifecycle Manager shows an alert banner: Problem occurred while connecting to the Lifecycle Manager server

Products

• VMware vSphere ESXi

Issue/Introduction

In the Lifecycle Manger Tab of the vSphere Client UI, you see:

Problem occurred while connecting to the Lifecycle Manager server.

Environment

vSphere vCenter Server 7.0.x
vSphere vCenter Server 8.0.x

Cause

The Update Manager service will not start and any attempt to patch vCenter Server using Update Manager fails.

Running the following you see that he vmware-updatemgr service is stopped:

service-control --status

Running:
 applmgmt lookupsvc lwsmd observability observability-vapi pschealth vlcm vmafdd vmcad vmdird vmonapi vmware-analytics vmware-certificateauthority vmware-certificatemanagement vmware-cis-license vmware-content-library vmware-eam vmware-envoy vmware-hvc vmware-infraprofile vmware-perfcharts vmware-pod vmware-postgres-archiver vmware-rhttpproxy vmware-sca vmware-sps vmware-statsmonitor vmware-stsd vmware-topologysvc vmware-trustmanagement vmware-vapi-endpoint vmware-vdtc vmware-vmon vmware-vpostgres vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-ui vstats vtsdb wcp
Stopped:
 vre-updatemgrmcam vmware-imagebuilder vmware-netdumper vmware-rbd-watchdog vmware-updatemgr vmware-vcha

When the service start is attempted, a certificate check transpires.  Checking the logging we see that failed with a permissions error:

less /var/log/vmware/vmware-updatemgr/refreshCerts-utility.log

[2024-07-31 08:34:27,673 INFO] Creating/Updating key file: /usr/lib/vmware-updatemgr/bin/ssl/rui.key
[2024-07-31 08:34:27,673 INFO] Creating/Updating certificate file: /usr/lib/vmware-updatemgr/bin/ssl/rui.crt
[2024-07-31 08:34:27,673 ERROR] Exception raised while refreshing the certs. err: [Errno 13] Permission denied: '/usr/lib/vmware-updatemgr/bin/ssl/rui.crt

Checking permissions on the folder and files at the location of the ruit.crt certificate we see that the file's permissions are incorrect, as they should all be the same:

ls -lah /usr/lib/vmware-updatemgr/bin/ssl/

root@sjc36-vctr [ ~ ]# ls -la /usr/lib/vmware-updatemgr/bin/ssl/
total 52
drwxr-xr-x  2 updatemgr updatemgr 4096 Nov 20  2023 .
drwxr-xr-x 10 root      root      4096 Sep 13  2023 ..
-rw-r-----  1 updatemgr updatemgr 4143 Nov 20  2023 jetty.crt
-rw-r-----  1 updatemgr updatemgr 1704 Nov 20  2023 jetty.key
-rw-r-----  1 updatemgr updatemgr    7 Nov 20  2023 KeyStorePassword
-rw-r-----  1 updatemgr updatemgr   40 Sep 13  2023 proxy.dat
-rw-r--r--  1 root      root      4143 Nov 20  2023 rui.crt
-rw-r-----  1 updatemgr updatemgr 1614 Nov 20  2023 rui.crt_Nov23_backup
-rw-r-----  1 updatemgr updatemgr 1704 Jul 31 08:34 rui.key
-rw-r-----  1 updatemgr updatemgr 4070 Nov 20  2023 vmware-vum.keystore
-rw-r--r--  1 updatemgr updatemgr 1253 Jun 19  2023 vua.crt

Resolution

To resolve, change the permissions on any file showing the wrong permissions.  For the rui.crt above:

chown updatemgr:updatemgr /usr/lib/vmware-updatemgr/bin/ssl/rui.crt

Check that the permissions are all correctly set to updatemgr:

ls -lah /usr/lib/vmware-updatemgr/bin/ssl/

root@sjc36-vctr [ ~ ]# ls -la /usr/lib/vmware-updatemgr/bin/ssl/
total 52
drwxr-xr-x  2 updatemgr updatemgr 4096 Nov 20  2023 .
drwxr-xr-x 10 root      root      4096 Sep 13  2023 ..
-rw-r-----  1 updatemgr updatemgr 4143 Nov 20  2023 jetty.crt
-rw-r-----  1 updatemgr updatemgr 1704 Nov 20  2023 jetty.key
-rw-r-----  1 updatemgr updatemgr    7 Nov 20  2023 KeyStorePassword
-rw-r-----  1 updatemgr updatemgr   40 Sep 13  2023 proxy.dat
-rw-r--r--  1 updatemgr updatemgr 4143 Jul 31  2023 rui.crt
-rw-r-----  1 updatemgr updatemgr 1614 Nov 20  2023 rui.crt_Nov23_backup
-rw-r-----  1 updatemgr updatemgr 1704 Jul 31 08:34 rui.key
-rw-r-----  1 updatemgr updatemgr 4070 Nov 20  2023 vmware-vum.keystore
-rw-r--r--  1 updatemgr updatemgr 1253 Jun 19  2023 vua.crt

Restart the updatemgr service:

service-control --start vmware-updatemgr

You can then confirm that the service has started by running the following command and ensuring it is listed in the "Running:" category of services.

service-control --status

Additional Information

This issue typically arises when the certificate files had been altered/edited in troubleshooting/resolving a previous issue.