vCenter Server Appliance backup over SCP stuck between 40 - 60%

Products

• VMware vCenter Server

Issue/Introduction

This article provides steps to resolve the backup issue when using SCP when the PNID of the vCenter Server Appliance is a short-name.

• VCSA backup using SCP gets stuck between 40 - 60% and fails
• Restoring the VCSA from such a backup fails with the error:

Metadata and system validation failed
Backup PNID is not resolved on the network please configure the network DNS service accordingly

• backup.log :

YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [Net::GetPNIDAddrInfo:Net.py:638] INFO: PNID '[Hostname]' is resolved with addresses: ['*.*.*.*']
YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [BackupManager::GenerateBackupMetadata:BackupManager.py:203] ERROR: PNID is not resolvable to any IP on theexisting system
YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [BackupManager::main:BackupManager.py:591] ERROR: BackupManager encountered an exception: name 'MetadataException' is not defined
YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [BackupManager::HandleBackupCleanup:BackupManager.py:469] INFO: Cleaning up the backup job.
YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [BackupManager::Cleanup:BackupManager.py:405] INFO: Cleaning up all running backup child processes.
YYYY-MM-DDTHH:MM:SS [MainProcess:PID-114541] [BackupManager::Cleanup:BackupManager.py:412] INFO: Cleaning up all sizeFiles.

• When executing the below command, the received output is not an FQDN or IP:

/usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost;

Environment

VMware vCenter Server Appliance

Cause

The backup process relies on the use of an FQDN or IP for the PNID. When a short-name is set for the PNID it is unable to resolve this correctly and the backup process fails.

Resolution

This is expected behavior as a result of deploying the VCSA using a short-name as the PNID.

For VCSA versions prior to 6.7 U3 it is not possible to change the PNID after deployment. For more info, see Cannot change the vCenter Server or Platform Service Controller 6.x hostname on versions prior to vCenter Server 6.7 Update 3.

Workaround:

To work around this issue, follow the steps below:

1. Connect to the console of the VCSA using SSH.
2. Take a backup of the files we will be editing using the command below:

cp /etc/resolv.conf /storage/core/resolv.conf.backup; cp /usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/util/Net.py /storage/core/Net.py.backup;

 

3. Add search domain.com to end of the /etc/resolv.conf file using the command below where domain.com the correct domain:

echo 'search domain.com' >> /etc/resolv.conf;
 

4. In the file /usr/lib/applmgmt/backup_restore/py/vmware/appliance/backup_restore/util/Net.py change the following line from:

ipV4Cmd = ['/usr/bin/dig', '+noedns', '+short', pnid, 'a']

to

ipV4Cmd = ['/usr/bin/dig', '+noedns', '+short','+search', pnid, 'a']

Note:

Ensure that only the +search parameter is added.
Net.py is read-only so need to save with :wq!

5. Restart the VAMI service using the command below:

service-control --stop applmgmt;service-control --start applmgmt;

5. Retry the VCSA backup via SCP.