vCenter upgrade fails at Stage 2 step 2 with error "A problem occurred while - Starting VMware Security Token Service"

Products

• VMware vCenter Server

Issue/Introduction

/var/log/firstboot/vmidentity-firstboot.py_xxxx_stderr.log

YYYY-MM-DDTXX:XX:XX.XXXZ  ReRegistering STS Using the lookup service
YYYY-MM-DDTXX:XX:XX.XXXZ  Further filtering retrieved service registration list on hostname : vcenter.domain.com.
YYYY-MM-DDTXX:XX:XX.XXXZ  Retrieved Service Registration Info of the local STS service: None
YYYY-MM-DDTXX:XX:XX.XXXZ  VMware Identity Service bootstrap failed.
YYYY-MM-DDTXX:XX:XX.XXXZ  Exception: Traceback (most recent call last):
  File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 1170, in main
    vmidentityFB.boot()
  File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 281, in boot
    self.reregisterSTSUsingLookupService()
  File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 435, in reregisterSTSUsingLookupService
    raise e
  File "/usr/lib/vmidentity/firstboot/vmidentity-firstboot.py", line 420, in reregisterSTSUsingLookupService
    raise Exception('Unable to find the sso endpoint for reregistering')
Exception: Unable to find the sso endpoint for reregistering

Environment

vCenter Server

Cause

Checking the cs.identity and sso:admin service registrations, we can see that the URL in the service registration is pointing to the vCenter IP address, whereas the hostname and PNID are pointing to the vCenter FQDN.

/usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://$HOSTNAME/lookupservice/sdk --type cs.identity | grep URL

/usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://$HOSTNAME/lookupservice/sdk --type sso:admin | grep URL

 

To check vCenter hostname and PNID:

hostname

/usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost

Resolution

1. Take valid snapshots and backup of the vCenter.

2. Run lsdoctor rebuild, referring to KB Using the 'lsdoctor' Tool

python lsdoctor.py -r

3. Restart the vCenter services.

service-control --stop --all
service-control --start --all

4. Check the service registration for the correct URL with the vCenter FQDN.

/usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://$HOSTNAME/lookupservice/sdk --type cs.identity | grep URL

/usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://$HOSTNAME/lookupservice/sdk --type sso:admin | grep URL

5. Upgrade the vCenter.