vSAN Data Protection 'Protection Groups' vanish after vCenter Certificates are replaced
search cancel

vSAN Data Protection 'Protection Groups' vanish after vCenter Certificates are replaced

book

Article ID: 401506

calendar_today

Updated On:

Products

VMware vCenter Server 8.0

Issue/Introduction

  •   After replacing vCenter certificate "Protection Groups" reflects as blank  under vSAN Data Protection, previously existing PG will not reflect.

  • Below message are seen in the vCenter UI
    <html> <head><title>502 Bad Gateway</title></head> <body> <center><h1>502 Bad Gateway</h1></center> <hr><center>nginx/1.25.2</center> </body> </html> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page --> <!-- a padding to disable MSIE and Chrome friendly error page -->

  • In vSAN Data Protection Appliance under storage /log/snapservice/snap-service-ui.log you see messages referring to FailedAuthentication

    {"level":"error","timestamp":"2025-06-17T16:07:41.662Z","C":"auth/client.go:359","message":"Failed to get token from cert/key pair","error":"ns0:FailedAuthentication: Password of the user logging on is expired. :: Password of the user logging on is expired. :: User account expired: {Name: com.vmware.vsan.snapservice, Domain:
     
  • In vCenter appliance under  /var/log/vmware/envoy/envoy-access.log you see messages referring to FailedAuthentication
    OPENSSL_internal:Connection timed out|33554464:system library:OPENSSL_internal:Broken pipe

    2025-06-17T15:13:31.794Z info envoy[61262] [Originator@6876 sub=connection] [Tags: "ConnectionId":"8376006"] remote address:#.#.#.#:60786,TLS_error:|33554542:system library:OPENSSL_internal:Connection timed out

    2025-06-17T15:13:31.794Z info envoy[61262] [Originator@6876 sub=connection] [Tags: "ConnectionId":"8376006"] remote address:#.#.#.#:60786,TLS_error:|33554542:system library:OPENSSL_internal:Connection timed out|33554464:system library:OPENSSL_internal:Broken pipe

    2025-06-17T15:13:31.795Z info envoy[61262] [Originator@6876 sub=connection] [Tags: "ConnectionId":"8375992"] remote address:#.#.#.#:60686,TLS_error:|33554542:system library:OPENSSL_internal:Connection timed out

    remote address:#.#.#.#:60786 --->will have an IP of the vSAN Data Protection Appliance

Environment

vCenter Server 8.x
vSAN Data Protection

Cause

vCenter username/password was missing from compose_snapservice_config.yaml file in Snapshot Service appliance.

/opt/vmware/snapservice/app/deployment/compose_snapservice_config.yaml

Resolution


1. Open an SSH session to the Snapshot Service appliance.

2. Stop snapservice  by running the command: 

systemctl stop snapservice


3. Edit /opt/vmware/snapservice/app/deployment/compose_snapservice_config.yaml and fill in the vCenter  - administrator.vsphere.local credentials.

If the domain name is not default vsphere.local, please update it as well.

Please also verify and confirm the VC and appliance addresses.

Please see a sample output below:

vsphereConfig:
useMockServer: false
vcenter: <vCenter FQDN> 
username: ""
password: ""
sslVerify: true
appliance: <Snapservice Appliance name> 
svcAccountCredentialPath: /secrets/svc_account_credential
domainName: vsphere.local

4. Start the snapservice  by running the command:

systemctl start snapservice

Additional Information

Using a ping test to test connectivity is not ideal as ICMP is blocked by default on the vSAN Data Protect Appliance so pining from vCenter to vSAN DP will fail, however pinging from the vSAN DP to vCenter will succeed if communication is working.

Test the communication between vCenter and the vSAN Data Protect Appliance by doing the following:

  1. Check the vSAN Data Protect UI services by running the below command via the vSAN DP CLI
    systemctl status nginx
  2. Try restarting the service by running
    systemctl restart nginx
  3. Check connectivity to the vSAN DP Appliance by entering the following commands from the vCenter console to bypass the proxy:
    wget -S https://<vSAN_DP_Applaince_Name>.example.com:8443/index.html --no-check-certificate 
    wget -S https://<vSAN_DP_Applaince_Name>.example.com:8443/main.js --no-check-certificate 
    wget -S https://<vSAN_DP_Applaince_Name>.example.com:8443/565.js --no-check-certificate 
    wget -S https://<vSAN_DP_Applaince_Name>.example.com:8443/38.js --no-check-certificate