VMware Live Recovery fails to reconfigure - Service 'snapservice' has not been imported
Article ID: 433326
Updated On:
Products
Issue/Introduction
You see these errors during both the initial fresh deployment and any subsequent RECONFIGURE or convergence tasks of the VLR Appliance
ERROROperation FailedA general system error occurred: N7Vmacore21InvalidStateExceptionE Service 'snapservice' has not been importedOperation ID: 27d9d399-9e48-4351-8d24-############2/27/26, 10:38:14 AM -0800
ERROROperation FailedReceived SOAP response fault from [<SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:26, <TCP '10.#.#.# : 51582'>, <TCP '10.#.#.# : 443'>>), /invsvc/vmomi/sdk>]: addSolutionRoleOperation ID: 7d740e45-84ae-4cff-98ef-############2/27/26, 10:39:45 AM -0800
Environment
VMware Live Recovery Appliance 9.x
Cause
This is caused due to a race condition within the VMware Directory Service (vmdir) and the Authorization Manager when operating in Enhanced Linked Mode (ELM).
When you perform a RECONFIGURE on the VLR Appliances simultaneously, it creates duplicate HmsAdmin roles. Because vCenter (SSO) replication is asynchronous, either sites (Protected or Recovery) don't know that a SolutionRole is being created parallely. Because of this race condition, vCenter ends up creating 2 distinct RoleIDs sharing the same RoleName thus breaking the unique name constraint that the va-configurator (VLR Appliance) relies on.
2026-02-27 10:48:26,012 [srm-reactive-thread-30] WARN com.vmware.dr.configservice.taskMonitor.ConfigureTaskHandler df479b0e-b9be-40c1-9453-############ getConfigureTaskProgress - Task finished with error!
(vmodl.fault.SystemError) {
faultCause = null,
faultMessage = null,
reason = N7Vmacore21InvalidStateExceptionE Service 'snapservice' has not been imported
}
at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(Unknown Source)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source)
at java.base/java.lang.reflect.ReflectAccess.newInstance(Unknown Source)
at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(Unknown Source)
at java.base/java.lang.Class.newInstance(Unknown Source)
va-config.log:
This confirms the Authorization Service is protecting itself from inconsistency, but quashing the RECONFIGURE process.
2025-12-15T10:08:45.944Z INFO va-config 2704 [VaConfig@4413 sub="vmomi.soapStub[218]" opID="21b83732-49f7-4887-ae7d-############-configure"] SOAP request returned HTTP failure; <SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:48, <TCP '44.#.#.# : 39034'>, <TCP '44.#.#.# : 443'>>), /invsvc/vmomi/sdk>, method: addSolutionRole; code: 500(Internal Server Error); fault: (dataservice.fault.AlreadyExistsFault) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = "Received SOAP response fault from [<SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:48, <TCP '44.#.#.# : 39034'>, <TCP '44.#.#.# : 443'>>), /invsvc/vmomi/sdk>]: addSolutionRole
--> "
--> }
2025-12-15T10:08:45.945Z WARNING va-config 2704 [VaConfig@4413 sub="cfg" opID="21b83732-49f7-4887-ae7d--############--configure"] VaConfigurator::Start():
--> (dataservice.fault.AlreadyExistsFault) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = "Received SOAP response fault from [<SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:48, <TCP '44.#.#.# : 39034'>, <TCP '44.#.#.# : 443'>>), /invsvc/vmomi/sdk>]: addSolutionRole
--> "
--> }
vpxd.log:
The specified principal ([email protected]) is invalid suggests that the SSO lookup mechanism is stumbling because the underlying identity store is in an inconsistent state due to the presence of duplicate roles.
2026-02-27T18:39:34.124Z info vpxd[06618] [Originator@6876 sub=vmomi.soapStub[351]] SOAP request returned HTTP failure; <<cs p:00007fa46000f920, TCP:localhost:1080>, /invsvc/vmomi/sdk>, method: getRoles; code: 500(Internal Server Error); fault: (dataservice.fault.NotAuthenticatedFault) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = "Received SOAP response fault from [<<cs p:00007fa46000f920, TCP:localhost:1080>, /invsvc/vmomi/sdk>]: getRoles
--> "
--> }
--> msg = "Received SOAP response fault from [<SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:28, <TCP '10.#.#.# : 58736'>, <TCP '10.#.#.# : 443'>>), /invsvc/vmomi/sdk>]: addSolutionRole
--> msg = "Received SOAP response fault from [<SSL(<io_obj t:N7Vmacore6System19TCPSocketObjectAsioE, h:28, <TCP '10.#.#.# : 58736'>, <TCP '10.#.#.# : 443'>>), /invsvc/vmomi/sdk>]: addSolutionRole
2026-02-27T18:39:34.224Z info vpxd[06994] [Originator@6876 sub=vmomi.soapStub[10] opID=24cf6163] SOAP request returned HTTP failure; <<cs p:00007fa46000f920, TCP:localhost:1080>, /sso-adminserver/sdk/vsphere.local>, method: findDirectParentGroups; code: 500(Internal Server Error); fault: (sso.fault.InvalidPrincipalFault) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>,
--> principal = "[email protected]"
--> msg = "Received SOAP response fault from [<<cs p:00007fa46000f920, TCP:localhost:1080>, /sso-adminserver/sdk/vsphere.local>]: findDirectParentGroups
--> The specified principal ([email protected]) is invalid.
--> Caused by: Principal cannot be found."
--> }
2026-02-27T18:39:34.225Z warning vpxd[06994] [Originator@6876 sub=SsoWrapper.SsoAdminFacade opID=24cf6163] [FindAllParentGroups] Cannot get direct parent groups of group Everyone vsphere.local. Exception N3Sso5Fault21InvalidPrincipalFault9ExceptionE(Fault cause: sso.fault.InvalidPrincipalFault
--> )
--> [context]zKq7AVECAQAAAG/ifgEfdnB4ZAAAQxxTbGlidm1hY29yZS5zbwAACBhCACk/QwCWmUoB/4IMbGlic3NvLXR5cGVzLnNvAAFMiAwC4WIhbGlidm1vbWkuc28AAuqQIQJfCiEC19oaAWGTDYNMr4UCdnB4ZACDgY6FAoNnYoUCg8rwhAIAXk89ADVRPQBNUT2DTZ/RAYNNDIMCAk11HIMU6WACg8yXgQKDF6iBAoMLu4ACg/qRgQIABOw3ABdFOADFD1EEsI4AbGlicHRocmVhZC5zby4wAAXf+g9saWJjLnNvLjYA[/context]
2025-12-15T10:08:45.927Z [authz-service-2 [] INFO com.vmware.cis.core.authz.accesscontrol.impl.PersistorImpl opId=a4c09bb0-e901-455b-a7f2-############] Updated role in Lotus store and cache succesfully 578731995 Name HmsAdmin-578731995
2025-12-15T10:08:45.927Z [authz-service-2 [] INFO com.vmware.cis.core.authz.accesscontrol.impl.AuthzServiceBaseImpl opId=a4c09bb0-e901-455b-a7f2-############] scheduling post role event on VC
2025-12-15T10:08:45.938Z [dataservice-7 [] INFO com.vmware.cis.core.authz.accesscontrol.impl.AuthorizationServiceInternalImpl opId=d5ce91de-76d5-4399-a7e6-############] [addSolutionRole] roleId = 1149; roleName = HmsAdmin
2025-12-15T10:08:45.941Z [dataservice-7 [] ERROR com.vmware.cis.core.authz.accesscontrol.impl.AuthorizationServiceInternalImpl opId=d5ce91de-76d5-4399-a7e6-############] [addSolutionRole] Solution role with name HmsAdmin already exists. Aborting.
Resolution
CAUTION: When managing vCenter Server in Enhanced Linked Mode (ELM), taking a normal snapshot is insufficient. Because vCenter instances in ELM constantly replicate a shared identity and configuration database, taking a snapshot of only one node while it is running can lead to a "USN Rollback" or database divergence if you ever need to restore it. If you take a snapshot of a running vCenter in ELM and later restore it, that node will have "old" data compared to its partners. Because the replication stream was never gracefully paused, the restored node may be rejected by its peers, effectively breaking your Linked Mode environment and requiring a complex manual cleanup or a full restore of all nodes to the same point in time.
To ensure data consistency across your entire SSO domain, you must perform a simultaneous, powered-off snapshot of all participating vCenter nodes.
NOTE: Take powered OFF snapshots of vCenter
1. Record the ESXi host on which vCenter resides
2. Set DRS to manual mode for the clusters in which the hosts reside
3. Shutdown the vCenters from their respective VAMIs
4. Once all nodes are shutdown, snapshot vCenter/s from the host client.
5. Power ON the vCenters
Now, you are good to move forward with the workaround.
When performing a RECONFIGURE on VMware Live Recovery (VLR) appliances in an Enhanced Linked Mode (ELM) environment, parallel execution causes a race condition in the Authorization Service. This leads to duplicate Solution roles and "AlreadyExists" faults.
Follow the workaround below to ensure a clean configuration.
OPTION 1:
This is the best practice for an initial deployment or a clean reconfiguration.
Site A (Production): Initiate the RECONFIGURE task on the VLR Appliance. Wait for the task to finish. Log into the vSphere Client and verify that the Solution Roles (HmsAdmin, SRM Administrator, etc.) are visible under Administration > Roles and that there are no duplicates present.
Wait an additional 15 minutes to allow the VMware Directory Service (vmdir) to replicate these new metadata objects to all partner vCenters in the SSO domain.
Site B (Recovery): Only after verification and the wait period, initiate the RECONFIGURE task on the second VLR Appliance.
OPTION 2:
Use this if a previous parallel RECONFIGURE attempt failed and the environment is now "dirty" with duplicate roles or stale registrations.
Service Registration Cleanup (vpxd MOB):
Navigate to the Managed Object Browser (MOB): https://<vcenter-fqdn>/mob/
Click on Content > ExtensionManager and UnregisterExtension com.vmware.vsan.snapshotservice
LDAP Cleanup (JXplorer):
Install JXplorer
Delete Service Registrations: com.vmware.dp
Delete Service Principals:
h5 (HTML5 UI components)
SRM / HMS
dpx-agents
snapservice
Ensure that no duplicate roles named HmsAdmin exist in the LDAP tree. If duplicates are found with different IDs, delete both to allow the next RECONFIGURE task to recreate a single, clean identity.
Follow the process OPTION 1 to RECONFIGURE the Appliances.
Feedback
