"External service 'EMC Backup Service' failed to respond in the specified timeout (10 SECONDS)" error when backing up vCloud Director with the Dell EMC Data Protection plugin

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• Backup of vCloud Director fails from the backup solution side.
• Messages similar to the following can be found in the  /opt/vmware/vcloud-director/logs/vcloud-container-debug.log:

WARN     | pool-jetty-24131185       | ExtensibilityFilter            | Unexpected failure while processing request URI /api/admin/extension/EmcBackupService/backupRepository/12345678-abcd-9101-efgh-12ab13cd14ef/backups

java.util.concurrent.TimeoutException: External service 'EMC Backup Service' failed to respond in the specified timeout (10 SECONDS)

Environment

VMware Cloud Director for Service Provider 9.x
VMware Cloud Director for Service Provider 10.x

Cause

This issue occurs when the backend solution is exceeding the default timeout set by vCloud Director.
The timeout is called extensibility.timeout and is set to 10 seconds by default.

Resolution

• To verify that the extensibility timeout of vCloud Director is set to 10 seconds, please run the following cell-management-tool command:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n extensibility.timeout -l

• Once you have confirmed that this is 10, then please run the following on the command line of vCloud Director to update the timeout to 30 seconds:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n extensibility.timeout -v 30

• Use the first cell-management-tool command to confirm that this has taken effect:

/opt/vmware/vcloud-director/bin/cell-management-tool manage-config -n extensibility.timeout -l