SDDC Manager Backup configuration failed with error - Invalid parameter: Validation failed for directory path

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

Unable to configure SDDC manager backup in site settings, receiving the below error:

Invalid parameter: Validation failed for directory path /storage/backup/sddcmanager/ on server 192.168.0.##. Please make sure backup directory is intact and sftp server has write permission on backup path.

 

You can find the error "End of IO Stream Read" in /var/log/vmware/vcf/operationsmanager/operationsmanager.log 

YYYY-MM-DDTHH:MM DEBUG [vcf_om,20190509c0c4abc,0509] [c.v.v.s.c.s.SecurityConfigurationServiceImpl,http-nio-127.0.0.1-7300-exec-4] Security config retrieved {"certificateValidationEnabled":true,"fipsMode":false}

YYYY-MM-DDTHH:MM ERROR [vcf_om,20190509c0c4abc,0509] [c.v.evo.sddc.common.util.SshUtil,http-nio-127.0.0.1-7300-exec-4] Unable to create jsch CLI session:

com.jcraft.jsch.JSchException: Session.connect: java.io.IOException: End of IO Stream Read

  at com.jcraft.jsch.Session.connect(Session.java:565)

  at com.vmware.evo.sddc.common.util.SshUtil.getSession(SshUtil.java:678)

  at com.vmware.vcf.secure.ssh.SshExecuter.<init>(SshExecuter.java:98)

  at com.vmware.vcf.secure.ssh.SshExecuterFactory.createSshExecuter(SshExecuterFactory.java:137)

  at com.vmware.vcf.secure.ssh.SshExecuterFactory$$FastClassBySpringCGLIB$$d3e8b1e0.invoke(<generated>)

  at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)

  at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)

  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)

Environment

VMware SDDC Manager 5.X

Cause

• The DEFAULT crypto-policy in RHEL9 systems does not allow SHA1.
• Incompatibility between SSH Server and SDDC Manager.
• This issue occurs in the following versions of the SFTP server. Newer versions may have the same issue.

  Open SSH version: 8.7.1 
  Guest OS: RHEL 9.2

Resolution

Currently, there is no resolution to this issue.

Workaround:

1. Add the following Key Algorithms to the bottom of the sshd_config file on the SFTP server.

   KexAlgorithms diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1

2. Restart the SSH service on the SFTP server.

Note:

When the operationsmanager.log shows the below error, then the workaround of updating the cryptographic policy to SHA1 on the affected RHEL9 server should be used:

 DEBUG [vcf_om,67ed261d8521124815f852a87e027b2c,427e] [c.v.v.s.c.s.SecurityConfigurationServiceImpl,pool-3-thread-9] Security config retrieved {"fipsMode":false}

ERROR [vcf_om,67ed261d8521124815f852a87e027b2c,427e] [c.v.evo.sddc.common.util.SshUtil,pool-3-thread-9] Unable to create jsch CLI session:
com.jcraft.jsch.JSchException: Session.connect: java.security.SignatureException: Bad signature length: got 404 but was expecting 384
        at com.jcraft.jsch.Session.connect(Session.java:565)

1. Apply the SHA1 subpolicy to the DEFAULT cryptographic policy: 

   update-crypto-policies --set DEFAULT:SHA1

2. Reboot the system 
3. Read more at Re-enabling SHA-1 

Note:

After successfully configuring the backup, the SHA1 submodule can be removed again from the RHEL server as it weakens the security of the system. The backup process will still work.