DR UI Slowness During Global Refresh and Navigation in Large-Scale Protection and Recovery Deployments

Products

• VMware Live Recovery

Issue/Introduction

Customers using Protection and Recovery 9.1 in large-scale deployments may observe that the DR UI becomes extremely slow or unresponsive when performing operations such as:

• Global Refresh
• Opening Recovery Plans
• Viewing Site Pair details
• Loading inventory after login

The issue is most noticeable in environments containing multiple DR appliances, several vCenter Servers, and hundreds of Protection Groups (particularly vSAN Snapshot Service Protection Groups).

Typical symptoms include:

• DR UI freezes for 30–60 seconds
• Recovery Plan details remain in a loading state
• Global Refresh causes the entire UI to become temporarily unresponsive
• Live HAR logs show most UI requests spending 99% of their execution time waiting for the backend (TTFB)

Environment

Protection and Recovery 9.1

Cause

The issue is caused by a scalability limitation that occurs in very large Protection and Recovery deployments due to the interaction between the Protection and Recovery DR UI and the vCenter Lookup Service during topology discovery operations. During a Global Refresh or similar topology-related operation, all DR appliances simultaneously perform synchronous Lookup Service (LSPP) discovery, generating a large burst of Lookup Service requests. Because the DR UI waits for all discovery requests to complete before updating the interface, delays in the Lookup Service directly impact UI responsiveness.

Although the vCenter Lookup Service continues to operate correctly and system resources such as CPU, memory, and LDAP performance remain healthy, the high volume of concurrent requests saturates the Lookup Service Tomcat worker threads, resulting in internal request queueing and serialization. Additionally, the Lookup Service CacheController serializes requests for identical cache keys, and the short cache lifetime causes frequent cache regeneration during heavy request bursts. This increases request contention and response latency, leading to slow or unresponsive UI behavior during Global Refresh operations in large-scale environments.

Resolution

Currently, a permanent product fix is not available. In the meantime, the workaround below can be implemented to improve overall UI performance. Please note that while this workaround improves general UI responsiveness, it does not mitigate the delay experienced during the Global Refresh operation. A permanent fix for this behavior is planned for a future release of Protection and Recovery.

Workaround:

1. Modify the vCenter's Lookup Service configuration (Both sites): /usr/lib/vmware-lookupsvc/web/conf/application.properties
   • ls.threadpool.stdexec.minSpareThreads=300
   • ls.cache.maxLife.sec=60
   • ls.rate.limiter.maxLocks=100

2. After updating the configuration, restart the vCenter's Lookup Service:

   • vmon-cli --restart lookupsvc