SDDC password remediation task for the backup user fails with "Update API for for BACKUP settings failed."
search cancel

SDDC password remediation task for the backup user fails with "Update API for for BACKUP settings failed."

book

Article ID: 404181

calendar_today

Updated On:

Products

VMware SDDC Manager

Issue/Introduction

  • The backup user will show up disconnected on the password manager, likely due to an expired password. 
  • After resetting the password for the backup user(if expired), the password will need to be remediated.
  • The remediation task will fail on the SDDC manager UI with the following error. 
  • Description
 Password remediate for resource :sddc_manage.domain.com, user : backup and credential type : FTP
Progress Messages Update API for for BACKUP settings failed.

Error

Message: Update API for for BACKUP settings failed.

Remediation Message:

Reference Token: 2H4DGF

Cause: Authentication failed on fileserver sftp://10.xx.xx.xx/nfs/vmware/vcf/nfs-mount/backup/cluster-node-backups/4.2.1.3.0.24533887-IPv4-20821742-2658-7bd8-bf50-5b1ef7cce599-1x.xx.xx.xx/tmpdir-2025-07-08T03_33_05UTC.
  • The /var/log/vmware/operationsmanger/operationsmanger.log will report the following ERROR message, which indicates that the NSX manager is unable to configure the backup on the sddc manager. 

YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,686cbd34f2b765279b002e5c34b02b57,aabb] [c.v.v.c.n.s.c.c.ApiConnection,pool-3-thread-14] Creating ApiClient to https://nsxt_manager_fqdn.domain.com.cn:443 with username admin
YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,686cbd34f2b765279b002e5c34b02b57,aabb] [c.v.v.c.n.s.c.c.ApiConnection,pool-3-thread-14] Created ApiClient connection to: nsxt_manager_fqdn.domain.com.cn
YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,686cbd34f2b765279b002e5c34b02b57,aabb] [c.v.v.c.n.s.c.c.ApiConnection,pool-3-thread-14] NSX Version: 4.2.1.3.0.24533884, NSX Version with policy baseline: 4.1.2.0.0-0
YYYY-MM-DDTHH:MM:SS+0000 DEBUG [vcf_om,686cbd34f2b765279b002e5c34b02b57,aabb] [c.v.v.c.n.s.c.c.ApiConnection,pool-3-thread-14] Overriding Mp handlers with policy handlers
YYYY-MM-DDTHH:MM:SS000+0000 DEBUG [vcf_om,686cbf98d68d17cff890100c9db2ebd4,9efa] [c.v.v.c.s.f.i.CertificateOperationsFacadeImpl,pool-9-thread-1] Clean validation result
YYYY-MM-DDTHH:MM:SS+0000 ERROR [vcf_om,686cbd34f2b765279b002e5c34b02b57,aabb] [c.v.v.c.n.s.c.c.ComplexHelpers,pool-3-thread-14] Exception occurred during NSX API invocation
java.util.concurrent.ExecutionException: com.vmware.vapi.std.errors.InvalidRequest: InvalidRequest (com.vmware.vapi.std.errors.invalid_request) => {
    messages = [],
    data = struct => {error_message=Authentication failed on fileserver sftp://10.xx.xx.xx:22/nfs/vmware/vcf/nfs-mount/backup/cluster-node-backups/4.2.1.3.0.24533887-IPv4-b7b51742-9e62-b5bb-b3cf-56df8b121016-1x.xx.xx.xx/tmpdir-YYYY-MM-DDT06_50_01UTC., httpStatus=BAD_REQUEST, error_code=29115, module_name=backup-restore},
    errorType = INVALID_REQUEST
}
        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
        at com.vmware.vapi.bindings.CompletionStageFuture.get(CompletionStageFuture.java:45)
        at com.vmware.vcf.common.nsxt.sdk.client.connection.ComplexHelpers.invoke(ComplexHelpers.java:188)
        at com.vmware.vcf.common.nsxt.sdk.client.connection.BackupOperations.update(BackupOperations.java:121)
        at com.vmware.vcf.common.nsxt.sdk.client.connection.NsxtManagerBackupOperations.configureBackupSchedule(NsxtManagerBackupOperations.java:37)
        at com.vmware.vcf.common.nsxt.services.impl.NsxtBackupServiceImpl.configureBackupSchedule(NsxtBackupServiceImpl.java:92)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.ConfigureNsxtBackupScheduleAction.lambda$execute$0(ConfigureNsxtBackupScheduleAction.java:200)
        at com.vmware.evo.sddc.common.util.RetriableCallable.call(RetriableCallable.java:182)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.ConfigureNsxtBackupScheduleAction.execute(ConfigureNsxtBackupScheduleAction.java:209)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.ConfigureNsxtBackupScheduleAction.execute(ConfigureNsxtBackupScheduleAction.java:41)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor1478.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85)
        at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71)
        at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.vmware.vapi.std.errors.InvalidRequest: InvalidRequest (com.vmware.vapi.std.errors.invalid_request) => {
    messages = [],

  • Manually updating the backup location from the NSX manager, pointing the SDDC manager as the backup server, will also fail with the message: 

sftp://10.xx.xx.xx:22/nfs/vmware/vcf/nfs-mount/backup/cluster-node-backups/4.2.1.3.0.24533887-IPv4-b7b51742-9e62-b5bb-b3cf-56df8b121016-1x.xx.xx.xx/tmpdir-YYYY-MM-DDT06_50_01UTC., httpStatus=BAD_REQUEST, error_code=29115, module_name=backup-restore}

 

Environment

VCF 5.x 

Cause

This issue can occur because the backup directory in the path /nfs/vmware/vcf/nfs-mount/backup has changed ownership or permissions. 

 

Resolution

To resolve the issue, modify the backup directory permissions and ownership in the path /nfs/vmware/vcf/nfs-mount/: 

  • The default ownership and permission of the directory should look like the following: 

drwxrwxr-x 5 vcf_lcm backup 4.0K Jul 14 07:44 backup

  • If the ownership or permissions are not as seen above, make the following changes

chown vcf_lcm backup /nfs/vmware/vcf/nfs-mount/backup/ 

chmod 775 /nfs/vmware/vcf/nfs-mount/backup/

  • Now re-attempt the backup password remediation. 
Powered by Wolken Software