Copy Cluster failed

Products

• Network Observability
• CA Performance Management

Issue/Introduction

You complete a cluster copy operation, but one or more nodes (for example, node-2) fail to start. When you check the bootstrap-catalog.log on the affected destination host, you see warnings indicating data corruption or missing bundle files:

2026-06-07 17:17:55.278 SAL Record Check Thread:0x7f9a7a7fc640 [SAL] <WARNING> Error getting size of bundle file of [####]: StorageBundleReader: Error reading metadata of file [/data/drdata/v_drdata_node0002_data/793/####_0.gt]: No files in the bundle
2026-06-07 17:17:55.278 SAL Record Check Thread:0x7f9a7a7fc640 <WARNING> @v_drdata_node0002: 01000/3938: MiniRos #### does not have proper SAL files

Environment

All supported NetOps Performance Management releases

Cause

This issue occurs due to localized data corruption on the destination node during or after the cluster copy process. The node cannot initialize because it cannot read the required Storage Abstraction Layer (SAL) metadata files.

Resolution

To resolve this issue, you must perform a forced startup of the corrupted node after the healthy nodes are already running. This allows the corrupted node to recover its data from its buddy nodes. NOTE: This is applicable only if 2 out of 3 nodes have started successfully post copy cluster.

 

• Start the healthy nodes normally. Ensure the nodes that do not show corruption are up and running to establish a quorum.

  • /opt/vertica/bin/vertica -D <database_directory> -C -n <node_name> -h <host_address> -p 5433 -P 4803

 

• Force start the corrupted node. Execute the startup command for the affected node using the -c flag. The -c flag instructs the node to clear its corrupted state and actively rebuild its data segments from its buddy nodes.

         Warning: The following command forces a node to recover data and should only be used when corruption is confirmed.

• • /opt/vertica/bin/vertica -D <database_directory> -C -n <node_name> -h <host_address> -p 5433 -P 4803 -c

 

• Monitor the recovery. Verify that the node joins the cluster and begins the synchronization process. You can check the node status with the following query:

  • SELECT node_name, node_state FROM nodes;