vSAN Snapshot plugin does not download after appliance deployment

Products

• VMware vSAN
• VMware Live Recovery

Issue/Introduction

Symptoms:

• vSAN Snapshot Service Appliance deployed as a VM successfully following Deploying vSAN Snapshot Appliance, but there is no plugin for the service deployed on the vSphere Client.
• Navigating to vSphere Client > vSAN cluster > Configure > Data Protection - It states that the Snapshot Service Appliance is not yet deployed.

Environment

• VMware vSAN 8.0.3.x
• VMware Live Recovery 9.0.3 and later
• VCF 9.x
• VMware vSAN 9.x
• Data Protection Virtual Appliance

Cause

• Since the Snapshot Appliance deployed successfully, the /storage/log/snapservice/snap-service.log on the appliance can be verified for cause of issue.

• The appliance fails to register to the vCenter and deploy the plugin due to failure to verify the vCenter certificate which was used during deployment of the vSAN Snapshot Appliance:

  {"level":"error","timestamp":"YYYY-MM-DDTHH:MM:SS.SSSZ","C":"auth/client.go:265","message":"Failed to initialize VC client","error":"Post \"https://<vCenter_FQDN>/sdk\": tls: failed to verify certificate: x509: certificate signed by unknown authority","S":"gitlab.eng.vmware.com/core-
.
.
• The cause of this verification failure is due to the certificate chain being incomplete.

Resolution

To resolve this issue:

1. Validate that the certificate on the Snapshot Appliance at path /etc/ssl/certs/snapservice/vc_certs.pem is same as the one from step - "Requisites before deploying Step 2" of deployment KB - Deploying vSAN Snapshot Appliance.

2. If it is not the same, then replace the contents of this file /etc/ssl/certs/snapservice/vc_certs.pem with the contents of the certificate from 'Begin' to 'End'. Then skip to Step 4. If this does not work, then perform Step 3 as well.

3. If the contents of the file /etc/ssl/certs/snapservice/vc_certs.pem and the certificate are same:

   1. In the SSH to the Snapshot Appliance run the command: openssl s_client -connect <vCenter_FQDN>:443

   2. In the output of this command, copy the "Server Certificate" content from "Begin" to "End".

   3. Take snapshot of the Snapshot Appliance.

   4. Then in the SSH to this appliance, delete the /etc/ssl/certs/snapservice/vc_certs.pem file.

   5. Create a /etc/ssl/certs/snapservice/leaf_cert.pem file and its content is the Server Certificate from the openssl command output (Step 3.2).

   6. Change the permissions of this file using the commands:

      • chmod 644 /etc/ssl/certs/snapservice/leaf_cert.pem

      • chown snapservice:snapservice /etc/ssl/certs/snapservice/leaf_cert.pem

4. Then restart the snapservice on the appliance using the commands:

   1. systemctl stop snapservice

   2. systemctl start snapservice

 

Check the vSphere Client now, a webpage refresh might be needed, the Data Protection service should be deployed successfully and the plugin visible as well.

 

If the plugin does not deploy even after performing above steps, validate the /storage/log/snapservice/snap-service.log on the Snapshot Appliance, if the below entries are seen:
"level":"error","timestamp":"YYYY-MM-DDTHH:MM:SS.SSSZ","C":"auth/client.go:213","message":"Failed to login to vc","error":"ServerFaultCode: Cannot complete login due to an incorrect user name or password."

Perform the below:

1. Update this file with the missing vCenter credentials (password & username) on the Snapshot Appliance - /opt/vmware/snapservice/app/deployment/compose_snapservice_config.yaml

2. Then restart the snapservice:

   1. systemctl stop snapservice

   2. systemctl start snapservice