Gathering VMware vRealize Automation log bundle may fail after a long uptime

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Gathering VMware vRealize Automation log bundle fails after a long uptime.

Note: The same behavior occurs either through the VMware vRealize Lifecycle Manager or directly through the vRA appliance using the vracli log-bundle command.

• In the log-bundle-yyyy-mm-dd.log, you see entries similar to

  2023-02-09 21:15:30,417 [ERROR] Will try to collect a smaller bundle by limiting the collected files: Not enough space on disk.

  Or

  No space left on device. Assembled bundle will be incomplete.

• Running the following command find /services-logs/ -type f -name "*.log.*.xz" | wc -l on the vRA appliance returns a value of 0.

Environment

VMware vRealize Automation 8.10.x
VMware vRealize Automation 8.11.x
VMware vRealize Automation 8.9.x
VMware vRealize Automation 8.8.x
VMware vRealize Automation 8.7.x

Cause

This issue occurs when the services log rotations are not being compressed after a long uptime. Uncompressed files make for a bigger log bundle which can lead to a bundle too big to be stored on disk. For more information on the cause, see Hang in bgp_delete.

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

Resolution

This is resolved in vRealize Automation 8.11.2.

Workaround:

Prerequisites

• Please take simultaneous non-memory snapshots of each virtual appliance(s) in the cluster.
• You have access to root user and password.
• You have SSH or console access to each virtual appliance.

Procedure

1. SSH / PuTTy into one vRA virtual appliance in the cluster.
2. Run this command to restart the monitor-services-log service:

   vracli cluster exec -- systemctl restart monitor-services-logs

Additional Information

Impact/Risks:
After a long uptime, gathering VMware vRealize Automation log bundle may fail and logs may be kept for a smaller period of time due to uncompressed logs taking up more space.