vSphere Replication REST API GET /api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vc-id}/vms, to retrieve VMs may return '503 Service Unavailable' if response exceeds one minute

Products

• VMware Live Recovery

Issue/Introduction

When requesting VMs through vSphere Replication API GET:/api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms, the API may return "503 Service Unavailable" and this is due to the vCenter server setup being large and dr-rest failed to finish the operation within one minute.

Example:

From dr-rest-api.* logs

For API GET /api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms, we will see similar error

2025-04-28 09:55:06,553 [srm-reactive-thread-35439] WARN  com.vmware.dr.restapi.infrastructure.requestHandlers.DrRestHandler ##################### ################# GET vms - Cannot process GET request to pairings/#################/vcenters/#################/vms:
com.vmware.dr.restapi.topology.impl.vmomi.invocation.ShutdownException: Client connection service for server cdlprecloudvc02.es.ad.adp.com has been stopped.

One minute later we will see "HTTP 503 Service Unavailable" in the same dr-rest-api.* log file.

2025-04-28 09:56:00,872 [jersey-background-task-scheduler-0] ERROR com.vmware.dr.restapi.infrastructure.jersey.errors.WebApplicationExceptionMapper - WebApplicationException error occurred javax.ws.rs.ServiceUnavailableException: HTTP 503 Service Unavailable

################# - {paringId} and {vCenterID}

Environment

VMware vSphere Replication 8.x

VMware vSphere Replication 9.x

Cause

In environments with a large number of VMs, this operation becomes time-consuming and may result in a timeout, causing the API GET/api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms/ call to fail with "503 Service Unavailable" .

Even with the field "filter_property=name&filter={vm-name}", dr-rest processes all VMs and then applies the filter, the API will still fail where the VMs are large.

 

Resolution

This optimization is currently targeted for our release with VCF 9.0. 

Workaround:

Let's try one of the following workarounds.

1. Try "suitable_for_replication=true" to optimize the API call. Like : /rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms?filter_property=name&filter={vm-name}&suitable_for_replication=True
    
2. If you knows the VMs that you want to replicate, we can use directly the API: GET /api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms/{vm_id}

Below is one method to find the {vm_id} using vSphere PowerCLI to fetch the {vm-id}:

 #Connect to vCenter server using Connect-VIServer command
 $vm = Get-VM -Name ###
 $vmId = $vm.Id # This is <id> above
 $vcId = $global:defaultviserver.InstanceUuid # This is <vcenter-id> above
 $vrRestVmId = -join("VirtualMachine:", $vmId.Substring(15), ":", $vcId)
  Write-Host $vrRestVmId

 vm_id format: "VirtualMachine:<id>:<vcenter-id>"
 vm_id example: "VirtualMachine:vm-56084:bb7d63f5-93dd-4430-b68d-0785ab3cbf4c"

Now you can call GET /api/rest/vr/v2/pairings/{pairing_id}/vcenters/{vcenter_id}/vms/VirtualMachine:vm-56084:bb7d63f5-93dd-4430-b68d-0785ab3cbf4c

### - Name of the VM