SDDC Backup Fails with Error - "Could not start SDDC Manager backup Backup failed : Unexpected error encountered when processing SDDC Manager Backup"

Products

• VMware SDDC Manager

Issue/Introduction

This article shares the steps to follow when the SDDC-Manager backup task fails with the below error:

On the operationsmanager.log , we see entries similar to below:

/var/log/vmware/vcf/operationsmanager/operationsmanager.log

YYYY-MM-DDThh:mm:ss.709+0000 DEBUG [vcf_om,d6038195bff####,ad6a] [c.v.v.b.helper.SosBackupApiClient,http-nio-127.0.0.1-7300-exec-8] Initiating backup from SOS API client. Backup spec: {"elements":[{"resourceType":"SDDC_MANAGER"}],"backupRetentionPolicy":{"retainNumOfMostRecent":5,"retainHourlyForDays":5,"retainDailyForDays":5}}
YYYY-MM-DDThh:mm:ss.343+0000 ERROR [vcf_om,d6038195bff####,ad6a] [c.v.v.b.helper.SosBackupApiClient,http-nio-127.0.0.1-7300-exec-8] An exception 500 INTERNAL SERVER ERROR: "{"arguments":[],"causes":[{"message":"Failed to get credentials for type : BACKUP_PASSPHRASE","type":null}],"context":null,"errorCode":"BACKUP_OPERATION_FAILED","message":"Unexpected error encountered when processing SDDC Manager Backup","referenceToken":"","remediationMessage":null}<EOL>" occurred while making a call to SOS
YYYY-MM-DDThh:mm:ss.343+0000 ERROR [vcf_om,d6038195bff####ad6a] [c.v.v.b.helper.SosBackupApiClient,http-nio-127.0.0.1-7300-exec-8] Response body as string {"arguments":[],"causes":[{"message":"Failed to get credentials for type : BACKUP_PASSPHRASE","type":null}],"context":null,"errorCode":"BACKUP_OPERATION_FAILED","message":"Unexpected error encountered when processing SDDC Manager Backup","referenceToken":"","remediationMessage":null}
YYYY-MM-DDThh:mm:ss.343+0000 ERROR [vcf_om,d6038195bff####,ad6a] [c.v.v.b.helper.SosBackupApiClient,http-nio-127.0.0.1-7300-exec-8] Sos client error message Unexpected error encountered when processing SDDC Manager Backup
YYYY-MM-DDThh:mm:ss.345+0000 DEBUG [vcf_om,d6038195bff####,ad6a] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7300-exec-8] Processing localizable exception Backup failed : Unexpected error encountered when processing SDDC Manager Backup
YYYY-MM-DDThh:mm:ss.346+0000 ERROR [vcf_om,d6038195bff####,ad6a] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7300-exec-8] [G2HDDJ] BACKUP_FAILED Backup failed : Unexpected error encountered when processing SDDC Manager Backup com.vmware.vcf.backuprestore.errors.BackupRestoreException: Backup failed : Unexpected error encountered when processing SDDC Manager Backup
    at com.vmware.vcf.backuprestore.translator.v1.ServiceToRestTranslator.toBackupsTask(ServiceToRestTranslator.java:115)
    at com.vmware.vcf.backuprestore.controller.v1.BackupRestoreController.triggerBackup(BackupRestoreController.java:243)
    at com.vmware.vcf.backuprestore.controller.v1.BackupRestoreController$$FastClassBySpringCGLIB$$6df05e03.invoke(<generated>)
 

 

 

Environment

VCF 4.x, 5.x

Cause

This is caused due to issues with the SOS client service

Resolution

Kindly follow the below steps to resolve the issue:

• Execute below command to restart SDDC services:

/opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh

• If the above doesnt help, proceed with rebooting the SDDC Manager.