"Failed to upload file from local storage." error when creating a backup archive schedule in Cloud Director Availability 4.x
Article ID: 315041
Updated On:
Products
VMware Cloud Director
Issue/Introduction
- Creating a backup archive schedule fails at the Server authentication step, and you see the following error:
Failed to upload file from local storage.
- In the /opt/vmware/h4/<cloud|manager|replicator>/log/<cloud|manager|replicator>.log file on the appropriate Cloud Director Availability appliance, you see messages similar to:
2023-05-11 09:17:47.544 ERROR - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : Unable to upload file /tmp/vcda_connection_test.
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : Exit code = 1
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : stdout = Enter password: Executing command:
/usr/bin/curl -T /tmp/vcda_connection_test sftp://##.##.##.##/mnt/DiskPool1/maitenance-backup/cama01/ --config -
Command stderr: ' % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (2) Failure establishing ssh session: -5, Unable to exchange encryption keys
'
Command exited with '2'
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : stderr =
2023-05-11 09:17:47.544 ERROR - [UI-########-####-####-####-############] [job-2] com.vmware.h4.jobengine.JobExecution : Task 448fbc8f-cfdb-4077-ba6a-d6bd8263e0b0 (WorkflowInfo{type='upload', resourceType='file', resourceId='null', isPrivate=false, resourceName='null'}) has failed
com.vmware.h4.exceptions.common.backup.FailedToUploadFileException: Failed to upload file from local storage.
at com.vmware.h4.common.backup.FileUploadDeleteJob.fail(FileUploadDeleteJob.java:156)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:125)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:37)
at com.vmware.h4.jobengine.JobEngine.execute(JobEngine.java:226)
at com.vmware.h4.jobengine.JobEngine.lambda$resumeQueuedJob$1(JobEngine.java:166)
at com.vmware.h4.jobengine.ThrottledExecutorService$1.run(ThrottledExecutorService.java:132)
at com.vmware.h4.common.mdc.MDCRunnableWrapper.run(MDCRunnableWrapper.java:30)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.vmware.subprocess.BadExitCodeException: Bad ExitCode: 1
at com.vmware.subprocess.ShellResult.checkExitCode(ShellResult.java:45)
at com.vmware.subprocess.Subprocess.exec(Subprocess.java:164)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:71)
... 8 common frames omitted
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : Exit code = 1
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : stdout = Enter password: Executing command:
/usr/bin/curl -T /tmp/vcda_connection_test sftp://##.##.##.##/mnt/DiskPool1/maitenance-backup/cama01/ --config -
Command stderr: ' % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (2) Failure establishing ssh session: -5, Unable to exchange encryption keys
'
Command exited with '2'
2023-05-11 09:17:47.544 DEBUG - [UI-########-####-####-####-############] [job-2] c.v.h.common.backup.FileUploadDeleteJob : stderr =
2023-05-11 09:17:47.544 ERROR - [UI-########-####-####-####-############] [job-2] com.vmware.h4.jobengine.JobExecution : Task 448fbc8f-cfdb-4077-ba6a-d6bd8263e0b0 (WorkflowInfo{type='upload', resourceType='file', resourceId='null', isPrivate=false, resourceName='null'}) has failed
com.vmware.h4.exceptions.common.backup.FailedToUploadFileException: Failed to upload file from local storage.
at com.vmware.h4.common.backup.FileUploadDeleteJob.fail(FileUploadDeleteJob.java:156)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:125)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:37)
at com.vmware.h4.jobengine.JobEngine.execute(JobEngine.java:226)
at com.vmware.h4.jobengine.JobEngine.lambda$resumeQueuedJob$1(JobEngine.java:166)
at com.vmware.h4.jobengine.ThrottledExecutorService$1.run(ThrottledExecutorService.java:132)
at com.vmware.h4.common.mdc.MDCRunnableWrapper.run(MDCRunnableWrapper.java:30)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.vmware.subprocess.BadExitCodeException: Bad ExitCode: 1
at com.vmware.subprocess.ShellResult.checkExitCode(ShellResult.java:45)
at com.vmware.subprocess.Subprocess.exec(Subprocess.java:164)
at com.vmware.h4.common.backup.FileUploadDeleteJob.execute(FileUploadDeleteJob.java:71)
... 8 common frames omitted
- Depending on the appliance role you are creating a backup schedule for, see the table below for the appropriate log file to check to confirm this issue:
| Appliance Role | Log Location |
|---|---|
| Cloud Director Replication Management Appliance | /opt/vmware/h4/cloud/log/cloud.log |
| Cloud Director Combined Appliance | /opt/vmware/h4/cloud/log/cloud.log |
| On-Premises to Cloud Director Replication Appliance | /opt/vmware/h4/replicator/log/replicator.log |
| vCenter Replication Management Appliance | /opt/vmware/h4/manager/log/manager.log |
| On-Premises to Cloud vCenter Replication Appliance | /opt/vmware/h4/manager/log/manager.log |
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware Cloud Director Availability 4.x
Cause
This issue occurs because the SFTP upload curl command uses libssh2 and in libssh2 the curl client cannot connect to OpenSSH servers running version 8.8 or later as OpenSSH Server 8.8 deprecated SHA1 signatures with RSA keys.
Resolution
This issue is resolved in Cloud Director Availability 4.7.
Workaround:
To work around this issue if you do not want to upgrade, perform the following steps on the external backup server when using OpenSSH server version 8.8 or later:
- Add the following lines to the server's sshd_config file.
HostKeyAlgorithms +ssh-rsa
PubkeyAcceptedKeyTypes +ssh-rsa
Note: The sshd_config file location depends on the operating system.
PubkeyAcceptedKeyTypes +ssh-rsa
Note: The sshd_config file location depends on the operating system.
- Then restart the sshd service, using a suitable command according to the underlying operating system.
Additional Information
- For more information, see the OpenSSH 8.8 Release Notes.
- For more information, see the libssh2 issue 1018.
Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.
Feedback
Yes
No
Powered by
