Log data transfer fails with error "Transfer failed due to an error" in VCF Operations 9.1.x

Products

• VCF Operations

Issue/Introduction

• When you attempt to perform a Log Data Transfer in VCF Operations 9.1.x, the process fails. The following error message appears in the user interface: 

  Transfer failed due to an error. Please try again.

   

• Reviewing the /var/log/loginsight/cassandra.log path on source cluster reveals specific database startup failures: 

  ERROR [HintsDispatcher:1] ####-##-##  HintsDispatchExecutor.java:243 - Failed to dispatch hints file ########-####-####-####-############-#############-#.hints: file is corrupted ({})

Environment

• Log Management 9.1.x
• VCF Operations for Logs 9.0.x
• Aria Operations for Logs 8.18.x
  
  

Cause

• This issue occurs because hints files are corrupted on the source cluster which causes the HintsDispatcher execution loop to fail.
• When this failure occurs, the Cassandra service crashes automatically on startup and prevents the cluster from reaching a healthy operating state.

Resolution

To resolve this issue, you must purge the corrupted hints data to restore database service stability:

1. Access the source cluster node(s) of your cluster via CLI as root user.
2. Take snapshot of the source VCF Operations for Logs cluster and follow the detailed steps to safely purge the corrupted files outlined in Error "Failed to dispatch hints file" "file is corrupted" in cassandra.log (315961)
3. Verify that the Cassandra service restarts successfully and the cluster status returns to healthy by running following command: 

   nodetool-no-pass status

   Note: It should return UN status for all the cluster nodes.
4. Retry your log data transfer operation.