Intermittent vCenter Backup Failures in ELM Environments: Server Validation Failure
search cancel

Intermittent vCenter Backup Failures in ELM Environments: Server Validation Failure

book

Article ID: 435987

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

In a vCenter Server Enhanced Linked Mode (ELM) environment, scheduled file-based backups intermittently fail for one or more nodes with the following symptoms:

  • The vCenter Server Appliance Management Interface (VAMI) reports: Backup job failed to start due to server validation failure.
  • Manual backup attempts may succeed, while scheduled jobs fail.
  • In /var/log/vmware/applmgmt/backupValidation.log: ERROR: Failed in backup connection validation: Plugin error occurred. ErrCode: 50, Args: ()
  • In /var/log/vmware/applmgmt/applmgmt.log (or rotated logs): ERROR:plugins.HttpStorageIOLib:MKDIR request failed. httpcode: 405, url: https://<backup-server>/.../ ERROR:plugins.HttpStorageIOLib:RMDIR request failed. httpcode: 207, url: .../ indicating a HTTP/1.1 500 Internal Server Error and HTTP/1.1 424 Failed Dependency.

Environment

vCenter 8.x

Cause

This issue occurs due to concurrent access conflicts on the backup target. When multiple vCenter nodes in an ELM environment attempt to validate the same backup location or create backup directories at the exact same time, the backup server may experience resource locking or session limits, returning a 405 (Method Not Allowed) or 500 (Internal Server Error).

Resolution

To resolve this issue, stagger the backup start times for each vCenter node in the ELM environment to prevent simultaneous validation requests.

  1. Analyze Current Schedules: Identify the start times for all vCenter nodes participating in the ELM.
  2. Modify Backup Schedules: Adjust the start time for each node to ensure they do not overlap during the initial connection validation phase.
    • Recommended Staggering: Introduce a minimum delay of 30 to 60 minutes between each vCenter node (e.g., Node 1 at 19:00, Node 2 at 19:30, Node 3 at 20:00).
  3. Verify Configuration: Log in to each vCenter VAMI (https://<vcenter-fqdn>:5480), navigate to Backup > Configure, and update the schedule.
  4. Monitor Logs: Confirm success in /var/log/vmware/applmgmt/backupScheduler.log after the next scheduled run.

Additional Information

Related Articles

VCSA Backup failed with error 'Failed to create backup directory on backup server'
Scheduled backup not taken for vCenter Server

Powered by Wolken Software