VMware Live Recovery error "Upstream connect error or disconnect/reset before headers. Reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused"

Products

• VMware Live Recovery

Issue/Introduction

Symptoms :

- srm-server service fails to start with backtrace in vmware-dr.log

/opt/vmware/support/logs/srm/vmware-dr.log :

2025-12-23T09:17:51.848+01:00 panic vmware-dr[07533] [SRM@6876 sub=Default] Application error: N7Vmacore9ExceptionE Failed to connect
 to remote services on ["#######.com"]
--> [context]zKq7AVECAAQAAL7KeAERdm13YXJlLWRyAABNBU9saWJ2bWFjb3JlLnNvAACiCSUA15kiAYl6AnZtd2FyZS1kcgABUQUMAXHeDAKe9QlsaWJkci1hdXRob3Jp
emF0aW9uLnNvAAEixQsBM5sMAQ6sCwF+WwMBO98MAxarDGxpYmRyLXZtYWNvcmUuc28AAbAHAwRKcgJsaWJjLnNvLjYABAVzAgFtEwM=[/context]
--> [backtrace begin] product: vmware-dr, version: 9.0.3.0, build: build-24693438, tag: vmware-dr, cpu: x86_64, os: linux, buildType:
 release
--> backtrace[00] libvmacore.so[0x004F054D]
--> backtrace[01] libvmacore.so[0x002509A2]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
--> backtrace[02] libvmacore.so[0x002299D7]: Vmacore::System::SystemFactory::CreateQuickBacktrace(Vmacore::Ref<Vmacore::System::Backt
race>&)
--> backtrace[03] vmware-dr[0x00027A89]
--> backtrace[04] vmware-dr[0x000C0551]
--> backtrace[05] vmware-dr[0x000CDE71]
--> backtrace[06] libdr-authorization.so[0x0009F59E]: Dr::Authorize::EntityWithPermissionsLifetimeMixin::_Activate(bool)
--> backtrace[07] vmware-dr[0x000BC522]
--> backtrace[08] vmware-dr[0x000C9B33]
--> backtrace[09] vmware-dr[0x000BAC0E]


- Reconfigure also fails with error :

ERROR
Operation Failed
A general system error occurred: N7Vmacore9ExceptionE com.vmware.vapi.std.errors.unauthorized

- vmware-dr.log has Credentials error 

/opt/vmware/support/logs/srm/vmware-dr.log :


 

2025-12-23T09:20:55.062+01:00 warning vmware-dr[12277] [SRM@6876 sub=LocalSite.LocalStsServer connID=sts-q3] Failed to connect
--> N7Vmacore9ExceptionE {
-->     "ERROR": {
-->         "com.vmware.vapi.std.errors.unauthorized": {
-->             "data": {
-->                 "OPTIONAL": null
-->             },
-->             "error_type": {
-->                 "OPTIONAL": "UNAUTHORIZED"
-->             },
-->             "messages": [
-->                 {
-->                     "STRUCTURE": {
-->                         "com.vmware.vapi.std.localizable_message": {
-->                             "args": [
-->                                 "Invalid Credential, VMware directory error[9234]"
-->                             ],
-->                             "default_message": "User is not authorized for (Invalid Credential, VMware directory error[9234])",
-->                             "id": "com.vmware.vcenter.svcaccountmgmt.unauthorized",
-->                             "localized": {
-->                                 "OPTIONAL": null
-->                             },
-->                             "params": {

- svcaccountmgmt.log on vCenter has Invalid credential error for the SRM account

/var/log/vmware/sso/svcaccountmgmt.log :

025-12-23T12:48:55.363Z ERROR svcaccountmgmt[79:tomcat-http--42] [CorId=8b882442-4dd0-4c3e-80eb-e5d7ea2a344e OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.PasswordMgmt] changeUserPassword: Got Directory Exception
2025-12-23T12:48:55.364Z ERROR svcaccountmgmt[79:tomcat-http--42] [CorId=8b882442-4dd0-4c3e-80eb-e5d7ea2a344e OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.DirectoryError] Invalid Credential, VMware directory error[9234]
2025-12-23T12:49:55.456Z INFO svcaccountmgmt[56:tomcat-http--19] [CorId= OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.PasswordMgmt] Changing Account Password for account: SRM-########@vsphere.local
2025-12-23T12:49:55.517Z ERROR svcaccountmgmt[56:tomcat-http--19] [CorId= OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.PasswordMgmt] changeUserPassword: Got Directory Exception
2025-12-23T12:49:55.517Z ERROR svcaccountmgmt[56:tomcat-http--19] [CorId= OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.DirectoryError] Invalid Credential, VMware directory error[9234]
2025-12-23T12:50:55.612Z INFO svcaccountmgmt[67:tomcat-http--30] [CorId=0b52b9db-bfb1-4821-8978-b68fb60f365d OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.PasswordMgmt] Changing Account Password for account: SRM-#########@vsphere.local
2025-12-23T12:50:55.663Z ERROR svcaccountmgmt[67:tomcat-http--30] [CorId=0b52b9db-bfb1-4821-8978-b68fb60f365d OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.PasswordMgmt] changeUserPassword: Got Directory Exception
2025-12-23T12:50:55.663Z ERROR svcaccountmgmt[67:tomcat-http--30] [CorId=0b52b9db-bfb1-4821-8978-b68fb60f365d OpId=] [com.vmware.vcenter.svcaccountmgmt.impl.DirectoryError] Invalid Credential, VMware directory error[9234]




Environment

VMware Live Recovery Manager 9.0.3.0

Cause

SRM service Account with credentials issue

Resolution

- Take snapshot of the vCenter

- Login vSphere Client -> Administration -> Users and Groups -> Groups -> Delete the SRM account "srm-xxxxx".

- Reconfigure Live Recovery