vSAN Data Protection Appliance Plugin failed to load with error "502 Bad Gateway"
search cancel

vSAN Data Protection Appliance Plugin failed to load with error "502 Bad Gateway"

book

Article ID: 444013

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

vSAN Data Protection Appliance Plugin failed to install on VMware vSAN 8.0.3 Express Storage Architecture (ESA) using vSAN data protection (Snap-Service) version 8.0.3.

  • Data protection appliance  boot screen show the events 'integrity: Problem loading X.509 certificate'

  • snap-service plugin failed to load on vCenter server reporting below error on UI.

    <body> <center><h1>502 Bad Gateway</h1></center> <hr>
    <center>nginx/1.25.2</center> </body> </html> <!-- a padding to disable MSIE and Chrome friendly error page -- > 

  • Upon verifying snap-service-panic.log on appliance, It is observed that the snap-service container is restarting with panic with event 'panic: Failed to initialize VC client'.

gitlab.eng.vmware.com/core-build/vsan_snapshot_service/pkg/server/vsphere/auth.initPersistentVCClient.func2()/build/mts/release/bora-24496861/snapservice/pkg/server/vsphere/auth/client.go:266 +0x11a
sync.(*Once).doSlow(0xc0005003a0?, 0xc000573ec0?) /build/mts/release/bora-24496861/compcache/cayman_go/ob-22775640/linux64/src/sync/once.go:74 +0xbf
sync.(*Once).Do(...)
        /build/mts/release/bora-24496861/compcache/cayman_go/ob-22775640/linux64/src/sync/once.go:65
gitlab.eng.vmware.com/core-build/vsan_snapshot_service/pkg/server/vsphere/auth.initPersistentVCClient({0x196ca30, 0xc000573ec0}, {{0x196ecc0?, 0xc00039ec90?}, 0x20001?}, 0xc00017ed10)
        /build/mts/release/bora-24496861/snapservice/pkg/server/vsphere/auth/client.go:258 +0x2ef
gitlab.eng.vmware.com/core-build/vsan_snapshot_service/pkg/server/vsphere/auth.Client.GetVimClient(...)
        /build/mts/release/bora-24496861/snapservice/pkg/server/vsphere/auth/client.go:79
        /build/mts/release/bora-24496861/snapservice/cmd/service/snapshot_service.go:30 +0x125
panic: Failed to initialize VC client

Environment

  • VMware vSAN 8.0.3 Express Storage Architecture (ESA)

  • vSAN Data Protection (8.0.3) 

Cause

  • The issue is caused by multiple DNS entries in the vCenter Certificate and snap-service unable to match the user provided VC hostname with the hostname being read from the VC certs retrieved from the SSL connection. This results in snap-service (data protection) container to crash.

  • snap-service.log  on vSAN Snap Service appliance show below events.

{"level":"info","timestamp":"2026-05-18T06:07:05.920Z","C":"fxevent/zap.go:122","message":"invoking","function":"gitlab.eng.vmware.com/core-build/vsan_snapshot_service/pkg/initializer.CreateServiceAccount()"}
{"level":"info","timestamp":"2026-05-18T06:07:05.921Z","C":"vcenter/service_accounts.go:265","message":"Sending create new svc account request..."}
{"level":"info","timestamp":"2026-05-18T06:07:05.921Z","C":"auth/client.go:711","message":"Creating new VC REST client"}
{"level":"error","timestamp":"2026-05-18T06:07:05.925Z","C":"auth/rest_client.go:40","message":"HTTP request failed with non-HTTP error;","srv":"https://####.###.com","path":"/rest/com/vmware/cis/session","query":"","args":null,"error":"Post \"https://####.###.com/rest/com/vmware/cis/session\": tls: failed to verify certificate: x509: certificate signed by unknown authority","S":"gitlab.eng.vmware.com/core-build/vsan_snapshot_service/pkg/server/vsphere/auth.

  • The DNS entries in the certificates can be verified by running the command below on vCenter server shell.

openssl s_client -connect <vCenter_FQDN>:443 </dev/null 2>/dev/null | openssl x509 -noout -text  | grep DNS

  • We see vCenter server certificate has two DNS entries as below.

DNS:###.###.com, DNS:####.###.com

Resolution

This is a known issue with vSAN data protection (snap-service) version 8.0.3 when vCenter certificate is using two DNS entries. The issue is fixed in vSAN Data Protection (snap-service) version 8.0.3.1. Build Number: 24755667

  1. vSAN Data Protect Appliance can be downloaded using the Download Link.

  2. Select Drivers and tools

  3. Search for VMware vSAN Snapshot Service Appliance 8.0U3a

  4. Download snapservice_appliance-8.0.3.1-24755667.iso(4.03 GB) Build Number: 24755667

Upgrading/deploying with vSAN Data Protection (snap-service) version 8.0.3.1 address the multiple DNS entries in vCenter certificate and allow the data protection (snap-service) container to stay stable. Once the snap-service container is healthy, the plugin would show up healthy.

Powered by Wolken Software