VLSR - Rate limit exceeded. Requests have been temporarily blocked
search cancel

VLSR - Rate limit exceeded. Requests have been temporarily blocked

book

Article ID: 445360

calendar_today

Updated On:

Products

VMware Live Recovery

Issue/Introduction

You are encountering HTTP 429 (Too Many Requests) errors while executing automation scripts that leverage the VMware Live Recovery (VLR) / Site Recovery Manager (SRM) PowerShell cmdlets and REST APIs. The issue is observed during bulk reporting and inventory collection operations, specifically when executing commands such as:

  • Invoke-SrmGetPairings
  • Invoke-SrmGetAllGroups
  • Invoke-SrmGetGroupVms

The script generate the following error:

Too many requests error calling GetPairings:
{"error_code":"4.1","message":"Rate limit exceeded. Requests have been temporarily blocked."}

 

Environment

VMware Live Site Recovery 9.0.5, 25103105

vCenter Server 8.0.3, 24853646

VMware ESXi 8.0.3, 24784735

Cause

The failures were caused by the VMware Live Recovery DR REST API rate limiting mechanism blocking requests after the configured per-session and/or per-IP request threshold was exceeded.

The PowerShell automation executes large numbers of SRM and VLR API requests while gathering replication and protection-group information.

The script performs nested loops across:

    • Site Pairings
    • Protection Groups
    • Protected VMs
    • Replication inventories

resulting in a large number of API calls being generated within a short time window. When multiple sequential REST API requests are generated within a short period of time, you hit this error because the affected cmdlets (Invoke-SrmGetPairings, Invoke-SrmGetAllGroups, Invoke-SrmGetGroupVms) all utilize the SRM/DR REST API endpoints protected by the built-in rate limiter.




Resolution

Option 1 

Increase the DR REST API per-session and per-IP rate limiter thresholds to accommodate the workload generated by the customer's automation.

Recommended values:

Per Session Limit: 200 - 300 requests
Per IP Limit: 200 - 300 requests

These values should be evaluated and adjusted according to your automation workload.


Option 2 (Script Enhancement)

Modify the automation to implement retry and backoff logic.

When a 429 response is received:

Wait 5-10 seconds
Retry the request

Because the limiter resets approximately 60 seconds after the first request, retry logic can prevent transient throttling failures without requiring significant configuration changes.


Option 3 (Script Enhancement)

Reduce API request volume by:

  • Caching previously collected data.

  • Avoiding repeated calls to the same endpoints.

  • Reducing nested API queries where possible.

  • Introducing request throttling within the script.

The DR REST API rate limiter was introduced as a security control to reduce the risk of denial-of-service (DoS) conditions caused by excessive API consumption.

Increasing the per-session and per-IP thresholds to approximately 200–300 requests is considered a reasonable adjustment for automation-heavy environments because additional protection mechanisms remain in place.

Specifically:

  • SRM Service remains protected by a global request limit.

  • vSphere Replication Service remains protected by a global request limit.

  • Config Service remains protected by a global request limit.

Each service continues to enforce a higher-level safeguard of approximately:

1000 requests per minute per service

across all sessions and IP addresses.

Therefore, increasing the session/IP thresholds does not remove protection against abusive traffic and remains an acceptable solution for legitimate automation workloads.

Powered by Wolken Software