Storage Replication Adapter (SRA) not found after upgrading SRM

Products

• VMware Live Recovery

Issue/Introduction

Symptom:

 

• SRA not found after upgrading SRM from 8.5 to 8.7.
  
  
• The SRA is Dell EMC RecoverPoint Storage Replication Adapter version 2.3.0.0
  
  
  
  
  
• Running Discover Devices fail with error : 
  
  'Storage adapter with UUID 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found'

 

 

Steps to Validate:

• /var/log/vmware/srm/vmware-dr.log

2025-03-12T22:41:49.288+08:00 warning vmware-dr[01025] [SRM@6876 sub=Storage opID=c5725784-24a8-4bf7-9137-7cfdb5df106e-retrievePropertiesEx:36b4] [52b64] Create: SRA 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' used by array manager 'storage-arraymanager-xxxx' was not found

2025-03-12T22:41:49.288+08:00 error vmware-dr[01025] [SRM@6876 sub=Storage opID=c5725784-24a8-4bf7-9137-7cfdb5df106e-retrievePropertiesEx:36b4]
 [52b64] GetAdapter: Storage adapter 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' was not found, error=
--> (dr.storage.fault.StorageAdapterNotFound) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = ,
-->    uuid = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
-->    msg = ""
--> }
--> [context]zKq7AVECAAQAAAFuSQEUdm13YXJlLWRyAADtoRxsaWJ2bWFjb3JlLnNvAAFdFitsaWJkci1zdG9yYWdlLnNvAAHo3xAB4fEaApC9bmxpYmRyLXR5cGVzLnNvAAMQIxlsaWJ2bW9taS5zbwADqisZA8tsGQOWkRkDxqIZA0lyGQOLsBcDqrIXA/zaFgOJaScEqykGbGliZHItdm1vbWkuc28AAE0BMQCtGDEADiVBBYd/AGxpYnB0aHJlYWQuc28uMAA=[/context]
--> [backtrace begin] product: VMware vCenter Site Recovery Manager, version: 8.7.0, build: build-21589505, tag: vmware-dr, cpu: x86_64, os: linux, buildType: release
--> backtrace[03] libvmacore.so[0x001CA1ED]
--> backtrace[04] libdr-storage.so[0x002B165D]
--> backtrace[05] libdr-storage.so[0x0010DFE8]
--> backtrace[06] libdr-storage.so[0x001AF1E1]
--> backtrace[07] libdr-types.so[0x006EBD90]

 

2025-03-12T21:41:11.076+08:00 verbose vmware-dr[01362] [SRM@6876 sub=Storage opID=9f69e71e-0abf-437e-bbc5-377fdc0e99db-reloadAdapters:d454] queryStrings responded with:
--> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
--> <Response xmlns="http://www.vmware.com/srm/sra/v2">
-->     <Strings locale="en">
-->         <String id="AdapterName">EMC VNX SRA</String>
-->         <String id="AdapterVendor">EMC</String>
-->         <String id="emcsra.could_not_find_device.desc">Could not find storage device.</String>
-->         <String id="emcsra.could_not_find_device.fix_hint">Check SRA logs and environment configuration.</String>
-->         <String id="emcsra.modify_file_system_mount_options.file_system_does_not_exist.desc">Could not modify file system mount options, no file system with this name exists.</String>
-->         <String id="emcsra.modify_file_system_mount_options.file_system_does_not_exist.fix_hint">Verify the file system to modify.</String>
-->         <String id="emcsra.modify_access.denied.desc">Device or group cannot be granted access to host(s) at this time.</String>
-->         <String id="emcsra.modify_access.denied.fix_hint">Resource may be in use.  Check the log files to verify device exists and retry operation.</String>
-->         <String id="emcsra.array_discovery.invalid_array_id.desc">Storage array could not be found with ID:</String>
-->         <String id="emcsra.array_discovery.invalid_array_id.fix_hint">Please check the log file for more information.</String>
-->         <String id="emcsra.modify.device_busy.desc">Device cannot be modified at this time.</String>
-->         <String id="emcsra.modify.device_busy.fix_hint">Resource may temporarily be in use.</String>
-->         <String id="emcsra.array_not_found.desc">The SRA was unable to find array from array identifier.</String>
-->         <String id="emcsra.array_not_found.fix_hint">Please verify storage environment, then collect logs and contact support for assistance.</String>
-->         <String id="emcsra.restore_checkpoint.unknown_error.desc">Could not restore file system checkpoint.</String>
-->         <String id="emcsra.restore_checkpoint.unknown_error.fix_hint">Please check the log file for more information.</String>
-->         <String id="emcsra.device_not_found.desc">The SRA was unable to find device from device identifier.</String>
-->         <String id="emcsra.device_not_found.fix_hint">Please verify storage environment, then collect logs and contact support for assistance.</String>
-->         <String id="emcsra.restore_checkpoint.does_not_exist.desc">Could not restore file system checkpoint, no checkpoint with this name exists.</String>
-->         <String id="emcsra.restore_checkpoint.does_not_exist.fix_hint">Verify the file system checkpoint to restore.</String>
-->         <String id="emcsra.connection.failure.desc">Unable to connect to storage array.</String>
-->         <String id="emcsra.connection.failure.fix_hint">Verify the array manager IP or hostname and user credentials.</String>
-->         <String id="emcsra.sync.invalid_peer_array_id.desc">Invalid peer array ID provided by SRM.</String>
-->         <String id="emcsra.sync.invalid_peer_array_id.fix_hint">Please check the logs for more information and verify storage environment.</String>
-->         <String id="emcsra.replicated_devices.not_found.desc">No replicated devices could be found.</String>
-->         <String id="emcsra.replicated_devices.not_found.fix_hint">Please verify the correct array managers and array pairs are configured.</String>
-->         <String id="emcsra.locale.not_supported.desc">Locale {Locale} not supported</String>
-->         <String id="emcsra.locale.not_supported.fix_hint">Collect logs and contact support for assistance.</String>
-->         <String id="emcsra.enabler_directory_not_found.desc">A compatible Enabler was not found.</String>
-->         <String id="emcsra.enabler_directory_not_found.fix_hint">Make sure one or more Enablers are installed in addition to the SRA.</String>
-->         <String id="emcsra.failover_replication_session.already_failed_over.desc">Could not failover replication session, it is already failed over.</String>
-->         <String id="emcsra.failover_replication_session.already_failed_over.fix_hint">Verify the failed over data is recent enough.</String>



2025-03-12T21:41:05.815+08:00 error vmware-dr[01031] [SRM@6876 sub=Storage opID=9f69e71e-0abf-437e-bbc5-377fdc0e99db-reloadAdapters:d454] Command queryInfo's stderr:
--> /srm/sra/command: line 18: can't create input.xml: Permission denied
--> org.apache.xmlbeans.XmlException: error: Unexpected end of file after null
-->     at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3471)
-->     at org.apache.xmlbeans.impl.store.Locale.parse(Locale.java:706)
-->     at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:690)
-->     at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:677)
-->     at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:208)
-->     at com.emc.santorini.xdo.request.CommandDocument$Factory.parse(Unknown Source)
-->     at com.emc.santorini.parse.SantoriniCommandParser.parseCommandXdo(SantoriniCommandParser.java:40)
-->     at com.emc.santorini.main.SantoriniMain.main(SantoriniMain.java:51)
--> Caused by: org.xml.sax.SAXParseException; systemId: file:; lineNumber: 1; columnNumber: 1; Unexpected end of file after null
-->     at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.reportFatalError(Piccolo.java:1038)
-->     at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:723)
-->     at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3439)
-->     ... 7 more
-->

 

• /var/log/vmware/srm/SRAs/SRAs/sha256_xxxxxxxxxxxxxxxxxxxxxxx/santorini.log

Mar 12, 2025 1:41:05 PM 1 WARNING: Unknown Error:
org.apache.xmlbeans.XmlException: error: Unexpected end of file after null
        at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3471)
        at org.apache.xmlbeans.impl.store.Locale.parse(Locale.java:706)
        at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:690)
        at org.apache.xmlbeans.impl.store.Locale.parseToXmlObject(Locale.java:677)
        at org.apache.xmlbeans.impl.schema.SchemaTypeLoaderBase.parse(SchemaTypeLoaderBase.java:208)
        at com.emc.santorini.xdo.request.CommandDocument$Factory.parse(Unknown Source)
        at com.emc.santorini.parse.SantoriniCommandParser.parseCommandXdo(SantoriniCommandParser.java:40)
        at com.emc.santorini.main.SantoriniMain.main(SantoriniMain.java:51)
Caused by: org.xml.sax.SAXParseException; systemId: file:; lineNumber: 1; columnNumber: 1; Unexpected end of file after null
        at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.reportFatalError(Piccolo.java:1038)
        at org.apache.xmlbeans.impl.piccolo.xml.Piccolo.parse(Piccolo.java:723)
        at org.apache.xmlbeans.impl.store.Locale$SaxLoader.load(Locale.java:3

Environment

VMware Site Recovery Manager 8.x

Cause

SRA (EMC RecoverPoint) docker fails to establish connection to the SRM service post the upgrade.

Resolution

1. Login to SRM appliance management page (https:<SRM-IP>:5480>) with Admin credentials.
   
   
2. Remove the SRA docker from 'Storage Replication Adapters' list and re-add it.
   
   
3. Return to Site Recovery UI page (https://<SRM-IP>/dr)
   
   
4. Under 'Storage Replication Adapters' section, click on 'Rescan Adapters'.
   
   
5. Go to 'Array Pairs' ,select the added Array pair and run 'Discover Devices'.
   
   
6. Validate status of 'Array Pair' and corresponding 'Replication LUNs'.