Upgrading ESXi hosts from SDDC manager fails on prechecks with the error "ESXi CLI checks for legacy boot mode and host filesystem have failed"

Products

• VMware SDDC Manager

Issue/Introduction

Symptoms:

• While running prechecks the SDDC manager reports an error when attempting to perform the legacy boot mode and filesystem symlinks CLI checks on the hosts:
  
  
  
  
• Hosts do not use legacy boot mode and no broken symlinks are to be found in the ESXi filesystem
• Operations manager logs shows errors similar to the excerpt below:
  
  operationsmanager.log
  ERROR [vcf_om,6719f51a97b5805d99b9df8d2ce46477,0ea7] [c.v.e.s.c.u.c.SshCommandExecuter,precheck-validation10] Could not connect to the SSH server @ esxi.domain.com for configuration.
com.jcraft.jsch.JSchException: timeout: socket is not established
        at com.jcraft.jsch.Util.createSocket(Util.java:394)
        at com.jcraft.jsch.Session.connect(Session.java:215)
        at com.vmware.evo.sddc.common.util.SshUtil.getSession(SshUtil.java:678)
        at com.vmware.evo.sddc.common.util.SshUtil.getSession(SshUtil.java:626)
        at com.vmware.evo.sddc.common.util.command.SshCommandExecuter.<init>(SshCommandExecuter.java:55)
        at com.vmware.evo.sddc.common.util.command.SshCommandExecuterFactory.createSshCommandExecuter(SshCommandExecuterFactory.java:71)
        at com.vmware.evo.sddc.common.util.command.SshCommandExecuterFactory.createSshCommandExecuter(SshCommandExecuterFactory.java:28)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)

Environment

VMware Cloud foundation 5.x

Cause

The prechecks are failing due to a communication problem of the SDDC manager with the ESXi hosts on port 22

Resolution

Steps :

1. Proceed to FIRST carry out a reboot of the ESXi host(s) that are being worked on. This will help to release any cached information, clear hung services, or reset blocked ports.

   • From the SDDC manager CLi to the ESXi Host(s)- test connectivity from the SDDC manager to the ESXi Host(s) 
     • ssh root@<ESXi Hosts IP> 
   • Run the Pre-checks again from the SDDC UI after all hosts are up and running.

2. If the issue persists, investigate network and TCP/UDP port connectivity issues on port 22 using Broadcom KB 341078

 

Additional Information

Network port requirements for VMware Cloud Foundation

 

In some occasion the issue might also occur due to incorrect/mismatched Host Keys stored in SDDC manager's known_hosts files.

Follow KB article 316028 to resolve the issue.