VCF Service Runtime Get Component Backups API Returns HTTP 500 Error Due to SFTP Directory Permission Issues

Products

• VCF Operations

Issue/Introduction

• When attempting to restore from a backup of the VCF Service Runtime, the UI returns the error:

Failed to fetch backups. API response for operation 'get component backups' is not a successful one.

• The VCF Service Runtime get component backups API endpoints (GET /api/v1/system/backups and GET /api/v1/components/{componentId}/backups) return HTTP 500 errors with permission denied messages in the VCF Service Runtime logs. Symptoms include:
  • API requests to list backups fail with HTTP 500 status
  • VCF Service Runtime logs show error:
    

    Error while walking directory tree: permission denied

    Collect log bundle to review logs. Service Runtime logs can be found under services-logs/vmsp-platform/{node-vm-name}/{vmsp-agent-pod-name}/vmsp-agent/console-log-{epoch-timestamp}.log.gz Build -> Lifecycle -> VCF Management -> Components

  • SFTP synthetic checks may still pass successfully
  • Backup creation workflows continue to work normally
  • The issue affects both system-level and component-level backup listing endpoints
  • Deployments using SFTP as the backup destination
  • Environments where the SFTP server has complex directory structures or mixed file permissions

All symptoms do not need to be present simultaneously, but the HTTP 500 error with permission denied in VCF Service Runtime logs is the primary indicator.

Environment

• VCF 9.1.0

Cause

The VCF Service Runtime list backups API uses a recursive directory traversal starting from the SFTP backup root directory (/backup by default). This traversal attempts to read all files and directories in the entire backup tree to build the backup listing response. Unlike targeted backup operations that only access specific paths, the list operation has "fail-fast" behavior—if any single file or directory in the traversed tree is unreadable due to permission restrictions, the entire API call fails with a permission denied error.

Resolution

Option 1: Fix SFTP Directory Permissions (Recommended)

1. Connect to the SFTP server as an administrator.
2. Navigate to the backup root directory (typically /backup).
3. Ensure the SFTP user account has the following permissions on all directories in the backup tree:
   • Read permission (r) on all files that need to be listed.
   • Execute/traverse permission (x) on all directories in the backup hierarchy.
   • This includes not just the backup files themselves, but any other files or directories that may exist in the backup root.
4. Use commands like chmod -R u+rx /backup (adjust as needed for your permission model) to ensure consistent permissions.
5. If using advanced access controls (ACLs, NFS identity mapping, SELinux/AppArmor), ensure these do not block the SFTP user's access to any part of the directory tree.

Option 2: Clean Up Backup Directory Structure

1. Remove any unnecessary files or directories from the backup root that are not part of the actual backup data.
2. Ensure only backup-related directories exist under the configured backup path.
3. This reduces the scope of the directory traversal and eliminates potential permission conflicts.

Option 3: Isolate Backup Directory (Advanced)

1. Configure a dedicated backup directory that contains only VCF backup data.
2. Update the SFTP configuration in the VCF Operations backup location configuration to point to this isolated directory.
3. This ensures the directory traversal only encounters files with consistent permissions.