Online Backups fail immediately when clicking Begin Backup Now

Products

• DX NetOps
• CA Spectrum

Issue/Introduction

Automatic backups do not run and when trying to manually run a backup by clicking the Begin Backup Now button on the VNM it immediately fails.

The events on the OnLineBackup model indicate a general failure.

Cause

The rcpd process was still running from a previous backup.  Most likely this was due to a Qualys scan.  Network issues can cause this too but are more rare.

Resolution

Kill the hung rcpd process.

You can run a process listing:

ps -ef | grep -i rcpd

and take a look at the run / start date.  If it is from an earlier time, kill the rcpd process.

Additional Information

As root/sudo you can run pstack on the rcpd process.  The stack will look similar to:

#0  0x00007fec617031bb in select () from /lib64/libc.so.6
#1  0x00007fec64de2e89 in CsBlockingCif::select_wait(fd_set*, fd_set*, fd_set*, timeval*, int*) () from /opt/SPECTRUM/lib/libcom.so.1
#2  0x00007fec64de2fa5 in CsBlockingCif::select_wait_read(int, int, int*) () from /opt/SPECTRUM/lib/libcom.so.1
#3  0x00007fec64de3679 in CsBlockingCif::rcvmsg(int, int*, unsigned int) () from /opt/SPECTRUM/lib/libcom.so.1