Vertica backup fails with "Another vbr task is currently running" error in CA Performance Management (CAPM)

Products

• Network Observability
• CA Performance Management

Issue/Introduction

The daily backup in CA PM DR server fails with the error

Unable to proceed.  Another vbr task is currently running: b'<SERVER_NAME>.3037412........................'.Hint: running concurrent vbr tasks with same snapshot name is not supported.
Backup FAILED.

Environment

DX NetOps CAPM all currently supported releases

Cause

Two jobs utilising the vbr.py utility are being run via cron at set times. The first has not finished when the second is executed by cron. The second fails to run as only one vbr.py instance can run at any one time.

In this scenario, there are two vbr log files generated:

vbr_20250809030002_TQU.log
vbr_20250809040003_WQD.log

vbr_20250809030002_TQU.log starts at 3am:

2025-08-09 03:00:02 localhost vbr VBR log initialized.

The other, vbr_20250809040003_7YIE0WQD.log at 4am:

2025-08-09 04:00:03 localhost vbr VBR log initialized.

Checking the cron jobs, the backup is run at 4am every day:

But the vbr.py Copycluster task is run at 3am. Looking at the end of the vbr_20250809030002_TQU.log file, the vbr.py Copycluster doesn't finish till 4:59am:

2025-08-09 03:00:51 192.168.0.3 vbr Opening backup location: rsync://[192.168.0.1]:50000/

2025-08-09 03:00:55 localhost vbr Determining what data to copy.

2025-08-09 03:01:00 localhost vbr Approximate bytes to copy: 37568622545 of 395698053095 total.

2025-08-09 03:01:00 localhost vbr Syncing data to destination cluster.

2025-08-09 04:59:09 localhost vbr Reinitializing destination catalog.

2025-08-09 04:59:09 localhost vbr Node v_drdata_node0001: bootstrapping catalog.

2025-08-09 04:59:28 localhost vbr Node v_drdata_node0001: catalog bootstrapped.

2025-08-09 04:59:28 localhost vbr Copycluster complete!

Even though it started at 3am. When looking at the other file, vbr_20250809040003_WQD.log, it tries to run the vbr.py in backupmode at 4am and fails:

2025-08-09 04:00:17 localhost vbr Error: On host 192.168.0.3: Unable to proceed.  Another vbr task is currently running: b'HOSTNAME.1699389.......'.Hint: running concurrent vbr tasks with same snapshot name is not supported.

  Backup FAILED.

So the problem in this scenario, is that sometimes, the vbr.py copycluster cron job doesn't finish within the hour, which stops the 4pm vbr.py backup job from starting.

 

Resolution

Put at least 2 hours between the two vbr.py cron jobs to fix this to allow adequate time for the first to finish.