SDDC Deployment ERROR [c.v.e.s.c.h.n.EsxiHostNetworkingUtil,pool-2-thread-12] Unable to ping #.#.#.# via VM Kernel vmk#

Products

• VMware SDDC Manager

Issue/Introduction

When deploying SDDC the deployment might fail if it cannot configure all required vm Kernel on the ESXI hosts. 

Environment

SDDC 4.5.x

SDDC 5.x 

Cause

SDDC Builds up the remaining required ESXI VM Kernel configuration as part of the deployment.  If it fails to configure these kernel so they are accessible then the deployment can fail. 

This is seen in the SDDC Standup debug log

[YYYY-MM-DDTHH:MM:SS] [bringup,684ac4d30190fd1065442963aaa97557,93d9] ERROR [c.v.e.s.c.h.n.EsxiHostNetworkingUtil,pool-2-thread-12] Unable to ping #.#.#.# via VM Kernel vmk30 with MTU 1470 from ESXi host XXXXXXXXXXXXX.XXX XXX
com.vmware.evo.sddc.common.client.vmware.esx.models.LocalizedMethodFault:  sendto() failed (Host is down)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
        at org.glassfish.jaxb.core.v2.ClassFactory.create0(ClassFactory.java:104)
        at org.glassfish.jaxb.runtime.v2.runtime.ClassBeanInfoImpl.createInstance(ClassBeanInfoImpl.java:255)

[YYYY-MM-DDTHH:MM:SS] [bringup,684ac4d30190fd1065442963aaa97557,93d9] ERROR [c.v.e.s.c.h.n.EsxiNetworkRetryableOperationsUtil,pool-2-thread-12] Unable to retrieve VMk vmk30 for ESXi Host XXXXXXXXXXXXX.XXX XXX
com.vmware.evo.sddc.common.hostservices.error.HostPreparationException: Failed to retrieve VMk vmk30 for ESXi Host XXXXXXXXXXXXX.XXX XXX
        at com.vmware.evo.sddc.common.hostservices.networking.EsxiNetworkRetryableOperationsUtil.lambda$retrieveVmk$1(EsxiNetworkRetryableOperationsUtil.java:50)
        at java.base/java.util.Optional.orElseThrow(Optional.java:403)
        at com.vmware.evo.sddc.common.hostservices.networking.EsxiNetworkRetryableOperationsUtil.retrieveVmk(EsxiNetworkRetryableOperationsUtil.java:46)
        at jdk.internal.reflect.GeneratedMethodAccessor1287.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)

Resolution

Review the ESXi vmnics to ensure they are correctly aligned with the corresponding physical NICs, allowing proper VMkernel configuration. The SDDC maps vmnics to physical NICs based on hardware IDs; therefore, any misconfiguration of the hardware NICs may result in alignment issues. Such misalignment can lead to communication failures for vmnics associated with the VMkernel, causing the VMkernel to fail ping tests and become unresponsive.

To resolve this:

1. Start the deployment over with aligned ESXI network cards

Or 

2. Customize the deployment inline for the impacted host correcting the alignment issue. This will result in uninformed ESXI host. However the deployment can be completed.