Best practices for backup and restore related issues on SFD
search cancel

Best practices for backup and restore related issues on SFD

book

Article ID: 323693

calendar_today

Updated On:

Products

VMware

Issue/Introduction

Symptoms:
Enabling backup and restore via CLI on SFD and verifying debug logs when issue arises

Resolution

Configuring and debugging Backup/restore related issues in SFD

sfd> backup location list


+---+-----------------+---------+------------------+----------+-----------------+---------------------+
| # | IP Address/FQDN |   User  | Backup Directory | Protocol |    Created By   |   Created At(UTC)   |
+---+-----------------+---------+------------------+----------+-----------------+---------------------+
| 1 |   10.79.xx.xx  | ftpuser |   /FTP/files     |   FTP    | [email protected] | 2021-01-18 10:30:12 |
+---+-----------------+---------+------------------+----------+-----------------+---------------------+



sfd> backup location add --host 10.79.xx.xx --user <username> --password <password> --backupdir /FTP/files --protocol ftp


sfd> backup job schedule --name backup_1 --content all --description first_backup
+---+-----------------+---------+------------------+----------+-----------------+---------------------+
| # | IP Address/FQDN |   User  | Backup Directory | Protocol |    Created By   |   Created At(UTC)   |
+---+-----------------+---------+------------------+----------+-----------------+---------------------+
| 1 |   10.79.xx.xx  | ftpuser |   /FTP/files     |   FTP    | [email protected] | 2021-01-18 10:30:12 |
+---+-----------------+---------+------------------+----------+-----------------+---------------------+

Please enter number of backup location that you want to store backup data: 1
Backup job scheduling status: BACKUP_JOB_OPERATION_SUCCESS



sfd> backup job list
+---+----------+---------+-------------------------------+------------+-----------------+-----------------+---------------------+
| # | Job Name |  Status |            Location           | Recurrence |     Content     |    Created By   |   Created At(UTC)   |
+---+----------+---------+-------------------------------+------------+-----------------+-----------------+---------------------+
| 1 | backup_1 | Expired | ftp://10.79.xx.xx/FTP/files  |   Never    |  Configurations | [email protected] | 2021-01-18 11:08:45 |
|   |          |         |                               |            | Events and Logs |                 |                     |
|   |          |         |                               |            | Monitoring Data |                 |                     |
+---+----------+---------+-------------------------------+------------+-----------------+-----------------+---------------------+



Below is an example of backup with logs

Initiating Backup  


2021/01/19 02:44:30.579 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.systemcontroller.backupandrestore.NfcBackupOrchestrator.initiateBackup(NfcBackupOrchestrator.java:429) - Start new backup activity
2021/01/19 02:44:30.579 [pool-6-thread-5] [DEBUG] com.vmware.nfc.platform.activitymanager.ActivityManager.createActivity(ActivityManager.java:146) - Received request to create new activity for name backup2 message Start taking new backup: 'backup2' for backup job: 'backup2_cem_19-04-2021_01-44-30.557.tar'. type NFC_ACTIVITY_TYPE_BACKUP subtype NFC_ACTIVITY_SUB_TYPE_CREATE_BACKUP status NFC_ACTIVITY_STATUS_IN_PROGRESS startTime 1618800270579 refOid oid: "208:53044847265649914"
 with 4 step
2021/01/19 02:44:30.586 [pool-6-thread-5] [DEBUG] com.vmware.nfc.platform.activitymanager.ActivityPublisher.updateAndPublishActivity(ActivityPublisher.java:51) - Receiving request to update oid {
  oid: "190:53044847266370811"
}
config {
  type: NFC_ACTIVITY_TYPE_BACKUP
  sub_type: NFC_ACTIVITY_SUB_TYPE_CREATE_BACKUP
  start_time: 1618800270579
  status: NFC_ACTIVITY_STATUS_IN_PROGRESS
  name: "backup2"
  message: "Start taking new backup: \'backup2\' for backup job: \'backup2_cem_18-01-2021_02-44-30.557.tar\'."
  service_name: "system-controller-service"
  current_step: -1
  total_step: 4
}
refs {
  back_refs {
    reference_oid {
      oid: "208:53044847265649914"
    }
  }
}
 activity with message: Starting backup... on BACKUP_ACTIVITY channel
2021/01/19 02:44:30.587 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.systemcontroller.backupandrestore.NfcBackupOrchestrator.initiateBackup(NfcBackupOrchestrator.java:435) - Set current backup instance status to IN_PROGRESS
2021/01/19 02:44:30.622 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.systemcontroller.backupandrestore.NfcBackupOrchestrator.initiateBackup(NfcBackupOrchestrator.java:440) - Start taking new backup
2021/01/19 02:44:30.622 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.systemcontroller.backupandrestore.NfcBackupOrchestrator.runBackup(NfcBackupOrchestrator.java:316) - contentTypes [CONFIG_DATA, EVENTS_AND_LOGS_DATA, MONITORING_DATA]



Completion of backup

2021/01/19 02:44:42.396 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.eventmanager.AbstractEventPublisher.publish(AbstractEventPublisher.java:111) - PUBLISH: Publish BACKUP_UPLOAD_SUCCESS_EVENT event on channel : BACKUP_UPLOAD_STATUS
2021/01/19 02:44:42.433 [pool-6-thread-5] [INFO ] com.vmware.nfc.platform.systemcontroller.backupandrestore.handler.BackupJobHandler.setBackupJobExpiry(BackupJobHandler.java:325) - Update the backup job state as COMPLETED



Restore Command

sfd>
backup restore initiate --filepath ftp://10.79.xx.xx/FTP/files/first-backup_ce_12-01-2021_06-18-25.571.tar  --username <usernmae> --password <Password>

Retrieving backup file info
Retrieved Backup File Attributes
+-------------------------------+---------------------------------------------+
| Name                          |                                       Value |
+-------------------------------+---------------------------------------------+
| File Name                     | first-backup_ce_12-01-2021_06-18-25.571.tar |
| File Size                     |                                       34 MB |
| File Modified Timestamp (UTC) |                         2021-01-12 06:16:42 |
+-------------------------------+---------------------------------------------+
Initiate restore request
Initiate restore status: RESTORE_INITIATED



Once restore is complete we should be able to see below messages

2021-01-19 03:17:46,421 [INFO ][services:sfd.common.services:verify_service_stop] - Pods have stopped!
2021-01-19 03:17:46,631 [INFO ][main:root:stop_app] - shut down maintenance app
2021-01-19 03:17:48,426 [INFO ][main:root:__start_services] - Stopped nginx in maintenance mode and maintenace app
2021-01-19 03:17:48,426 [INFO ][services:sfd.common.services:start_service] - Starting service /opt/vmware/nfc/scripts/deployment/nginx-gw.yaml
2021-01-19 03:17:48,587 [INFO ][services:sfd.common.services:verify_service_start] - Verifying pods startup for service label[application=nginx-gw]
2021-01-19 03:17:48,697 [INFO ][services:sfd.common.services:verify_service_start] - Pods are not ready. Validating after 15 secs
2021-01-19 03:18:03,821 [INFO ][services:sfd.common.services:verify_service_start] - Pods are ready!
2021-01-19 03:18:03,822 [INFO ][main:root:init_restore] - ---------------restore success!-----------------
2021-01-19 03:18:03,822 [INFO ][main:root:init_restore] - Nginx started




During backup or restore if there are any failures we will need to collect the logs in debug and can verify in the below location
  • Backup logs can be found in the system-controller-service and NFC-host-service
  • Restore logs can be found in system-controller-service, NFC-host-service and restore.log file