"OutOfMemoryError: GC overhead limit exceeded" error on offline Replicators in vCloud Availability

Products

• VMware Cloud Director

Issue/Introduction

• Creating new protections or migrations fail in vCloud Availability.
• Performing a test failover, failover, or migration for active replications fail.
• In the System Monitoring view of the vCloud Availability Portal, the Replicator show as offline.
• When navigating to the Replicator Management Portal, it is not available.
• In /opt/vmware/h4/replicator/log/replicator.log on the Replicator, you see entries similar to:

WARN - [dd3aba8a-434a-4948-8b98-a5fd18d9a9f4_PU] [https-jsse-nio-8440-exec-10] c.v.h.c.e.ExceptionConversionService     : Unable to convert exception. Using fallback exception instead. org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: GC overhead limit exceeded

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vCloud Availability 4.x

Cause

This issue occurs when the Replicator service spends most of its time performing garbage collection causing the service to become unresponsive.

Resolution

Workaround:

To work around this issue, you have to restart the Replicator service.

1. SSH to the Replicator appliance and login as root.
2. Check the status of the Replicator service:

systemctl status replicator.service

3. If you see GC overhead limit messages in the output, restart the Replicator service:

systemctl restart replicator.service

4. Check the status of the Replicator service again to ensure it has been restarted successfully.
5. Navigate to the Replicator Management Portal to verify it is available again via the URL:

https://replicatorIp:8440/ui/admin

6. If the Management Portal is still not available, restart the Replicator appliance.