vrUi Service using expired SSL Trust certificate found on vCenter Server

Products

VMware vCenter Server

Issue/Introduction

Expired certificates detected to be associated with a service registration in the vCenter Server
The service type for the expired certificate was found to be vrUi
There are no expired certificates in the VECS store on the vCenter as identified with the command : for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;

Potential error message in the GUI:
Unable to retrieve pairs from extension server at https://vspherereplication_IP.local:8043.A generic error occurred in the vSphere Replication Management Server. Exception details: 'Error creating bean with name 'com.vmware.hms.remote.HmsServerImpl': Unsatisfied dependency expressed through field 'repositoryFactory'; nested exception is org.springframework.beans.factory.BeanCreationNotAllowedException: Error creating bean with name 'repositoryFactory': Singleton bean creation not allowed while singletons of this factory are in destruction (Do not request a bean from a BeanFactory in a destroy method implementation!)'.

Environment

vCenter Server with SRM integration

Cause

This can be because of previously registered services that were not cleaned up from the vCenter Server while registering services with updated certificates.

Resolution

Take proper snapshots of the vCenter Server before proceeding.
Snapshot Best practices for vCenter Server Virtual Machines

Verification

The below command will show that there are more than 2 Service Registrations for the vrUi ( SRM ) service by identifying the entries with "Service Type: vrUi"
/usr/lib/vmware-lookupsvc/tools/lstool.py list --url http://localhost:7090/lookupservice/sdk

Check these service registrations from the output generated on the above command and make note of the "Service ID" field and the "SSL Trust" field.
An entry for the Service registration would have the below fields.

Name: VMware Site Recovery
Description: Site Recovery Web Client Plugin
Service Product: com.vmware
Service Type: vrUi
Service ID: h5-dr-########-####-####-####-c0########84
Site ID: ho
Owner ID: 5-dr-########-####-####-####-c0########84@domain.com
Version: 8.#.#.#

SSL Trust:
MIIGwjCCBa............................................................................................................................
............................................................................................................................uZ292LmF1

Decode the "SSL trust:" section using the base64 command or any other tool.

- Using base64 command :
  - - export CERTCHECK="MIIGwjCCBa.................................................uZ292LmF1"
    - echo "$CERTCHECK" | base64 -d | openssl x509 -noout -text
Identify the SSL trust: fields that have an expired certificate and note the "Service ID: " for the same.
For example : Service ID: h5-dr-########-####-####-####-c0########84

Removal :

Remove the service registration from the vCenter Server using the below command :
/usr/lib/vmware-lookupsvc/tools/lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id <Service ID> --user '<vCenter Administrator User>' --password '<vCenter administrator password>' --no-check-cert

Example :
/usr/lib/vmware-lookupsvc/tools/lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id h5-dr-########-####-####-####-c0########84 --user 'administrator@domain.local' --password 'Passw0rd!' --no-check-cert
Restart the vCenter Server services on vCenter Servers in the SSO domain using command :
service-control --stop --all && service-control --start --all ;
Verify if the service has been successfully deleted using the command :
/usr/lib/vmware-lookupsvc/tools/lstool.py list --url http://localhost:7090/lookupservice/sdk | grep -ia "< Service ID >"

Additional Information

Cleaning up decommissioned SRM registration from vCenter Server