Error occurred while fetching machine certificates

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:
1) When using custom machine certificates, and accessing administration -> certificate management and we see below error:


2) When reviewing vsphere_client_virgo.log

2021-09-10T14:34:01.466Z] [ERROR] http-nio-5090-exec-905    com.vmware.vise.mvc.exception.GlobalExceptionHandler       Exception handled while processing request for /ui/certificate-ui/ctrl/certificates/vmca-root?endPoint=vCENTER_FQDN: com.vmware.vapi.std.errors.Unauthenticated: Unauthenticated (com.vmware.vapi.std.errors.unauthenticated) => {
messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
  id = com.vmware.vapi.endpoint.method.authentication.required,
  defaultMessage = Authentication required.,
  args = [],
  params = <null>,
  localized = <null>
}],
  data = <null>,
  errorType = UNAUTHENTICATED,
  challenge = Basic realm="VAPI endpoint",SIGN realm=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,service="VAPI endpoint",sts="https://vCENTER_FQDN/sts/STSService/vsphere.local"

Environment

VMware vCenter Server 7.x

Cause

This generally happens when STS service is not able to authenticate the processing for "/ui/certificate-ui/ctrl/certificates/vmca-root?endPoint"

Resolution

Take offline snapshots of vCenter prior to proceeding.  If this is an ELM configured vCenter take offline snapshots of all vCenter participating in ELM. 

1. Download fixsts.sh script This script can be downloaded from KB Ref: "Signing certificate is not valid" or "No healthy upstream" error in vCenter Server Appliance
2. Upload fixsts.sh script to vCenter Server /tmp folder using WinScp.
3. If the connection to upload to the vCenter by the WSCP client is rejected you may need to change the shell Refer to Toggling the vCenter Server Appliance default shell
   
   
4. Navigate to the /tmp directory
   
   cd /tmp 


5. Make the file executable.
   
   chmod +x fixsts.sh


6. Run ./fixsts.sh 
7. Restart services on vCenter using below commands
   
   service-control --stop --all
service-control --start --all
   
   
8. Real time execution of services can be monitored by running below command on separate SSH session of same vCenter server
   
   watch service-control --status --all

Additional Information

Please Note: After running fixsts.sh successfully, it might take time to sync all certificates and if we check certificate management from vCenter web console we might see below error:


Under such condition it is recommended to reboot the vcenter server once.