Error: "use of closed network connection" and "SSH_FX_FAILURE" when configuring SFTP backup
search cancel

Error: "use of closed network connection" and "SSH_FX_FAILURE" when configuring SFTP backup

book

Article ID: 439957

calendar_today

Updated On:

Products

VCF Operations VCF Automation

Issue/Introduction

The VCF Automation 9.0.1 SFTP backup task fails, which blocks an Apply Patch operation in Fleet Management. You may experience multiple failure stages and errors during configuration or execution:

  • The manual backup task in Fleet Management fails with Error Code: LCMVMSP10010.

  • Initially fails with a use of closed network connection error.

  • Transitions to an SSH_FX_FAILURE: Folder not found error following an SFTP server upgrade.

  • Triggers a ValidateSyntheticCheckerException: connection reset by peer error when attempting to update credentials in the UI.

To verify the specific failure points, perform the following:

  • Run the appliance shell command sftp -vvv <sftp_username>@<SFTP_SERVER_IP> to observe the Key Exchange (KEX) negotiation. The output confirms the rejection:

Unable to negotiate with [sftp_IP] port 22: no matching host key type found. Their offer: ssh-dss,ssh-rsa

  • Review the failed SFTP backup task to observe remote file system operation errors resulting from absolute pathing (/vcf9_poc/) and directory creation failures:

ERROR : YYYY-MM-DDTHH-MM-SSZ.backup.tgz: Post request rcat error: Put mkParentDir failed: mkdir dirExists failed: dirExists stat failed: sftp: "Folder not found: /users/[userName]/vcf/backups/flt-auto/9.0.1.0/vcfa/YYYY-MM-DDTHH-MM-SSZ" (SSH_FX_FAILURE)

Failed to rcat with 2 errors: last error was: Put mkParentDir failed: mkdir dirExists failed: dirExists stat failed: sftp: "Folder not found: /users/[userName]/vcf/backups/flt-auto/9.0.1.0/vcfa/YYYY-MM-DDTHH-MM-SSZ" (SSH_FX_FAILURE)

ERROR - Upload from s3:backup/vcf/backups/flt-auto/9.0.1.0/vcfa/YYYY-MM-DDTHH-MM-SSZ/YYYY-MM-DDTHH-MM-SSZ.backup.tgz to sftp://[userName]@[FTP_SERVER_FQDN]/users/[userName]/vcf/backups/flt-auto/9.0.1.0/vcfa/YYYY-MM-DDTHH-MM-SSZ/YYYY-MM-DDTHH-MM-SSZ.backup.tgz has failed"

  • Observe the UI error ValidateSyntheticCheckerException when saving settings.

  • Review Fleet Management logs (/var/log/vrlcm/vmware_vrlcm.log) for an invalid double-slash URI:

"backup_path":"sftp://user@domain/vcf_lab//vcf/backups/flt-auto/9.0.1.0/..."

Environment

VCF Operations 9.0.x
VCF Fleet Management 9.0.x
VCF Automation 9.0.x

Cause

The backup task failures are caused by a combination of the following factors:

  • The legacy target SFTP server supports deprecated key exchange algorithms (ssh-dss, ssh-rsa) that the OpenSSH client strictly rejects.

  • The configured absolute path violates user permissions, or the target SFTP server has limitations preventing it from automatically creating the full directory path required to match the VCFA local backup location.

  • The Fleet Management UI payload is rejected by the vmsp-platform pod due to a conflicting cache within the internal Kubernetes cluster.

  •  A trailing slash applied to the base directory configuration (vcf_lab/) creates an invalid double-slash anomaly, causing SFTP daemons to reject the traversal request.

 

Resolution

To resolve the SFTP backup failures, perform the following steps:

  1. Upgrade the target SFTP server to support modern OpenSSH cryptographic standards (e.g., aes256-gcm@openssh.com, ecdsa-sha2-nistp256).

  2. Update the backup directory configuration to use a relative path (e.g., vcf_lab) and remove any trailing slashes.

  3. Manually construct the required target directory structure on the SFTP server using the VCF appliance's interactive sftp shell to bypass the server's directory creation limitations:

    1. Log in to the Fleet Management appliance as root via SSH.

    2. Run the following command to connect to the target SFTP server:

      sftp [userName]@[SFTP_Server_fqdn]

      (Note: Replace [userName] with the configured SFTP service account set in the Fleet Management Backup settings, and replace [SFTP_Server_fqdn] with the FQDN or IP of your target SFTP server).

    3. Create the necessary nested directory structure to match the path expected by the backup task.

  4. Manually update the vmsp-platform pod deployment via the kubectl patch command to resolve the UI synthetic check errors. For detailed steps, see Cannot configure SFTP backup destination in VMware Aria Operations due to error LCMVMSP10035 (432018) or Unable to configure SFTP backups after username and password change in Fleet Manager (416350).

  5. Use the Fleet Management API to forcefully terminate the stale task lock preventing the Apply Patch execution. For instructions, see Cancel long running Fleet Management Lifecycle requests (415510).

Powered by Wolken Software