Add Host to Cluster fails on Management Portgroup Uplink Teaming Validation Action

Products

VMware SDDC Manager

Issue/Introduction

When an ESXI Host with a mixed-case FQDN is added to a cluster ManagementPortgroupUplinkTeamingValidationAction, it fails with an error similar to:

/var/log/vmware/vcf/domainmanager.log

VCF_ERROR_INTERNAL_SERVER_ERROR Invocation of prefix '' part of task ManagementPort
groupUplinkTeamingValidationAction in plugin HostNetworkingPlugin failed with exception.
com.vmware.evo.sddc.common.core.error.InternalServerErrorException: Invocation of prefix '' part of task ManagementPortgroupUplinkTeamingValidationAction in plugin HostNetworkingPlugin failed with exception
.
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:447)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor384.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:88)
        at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:73)
        at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:68)
        at org.springframework.cloud.sleuth.instrument.async.TraceRunnable.run(TraceRunnable.java:68)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)Caused by: java.lang.NullPointerException: null
        at com.vmware.vim.vmomi.client.http.impl.HttpClient.resolveMoRef(HttpClient.java:124)
        at com.vmware.vim.vmomi.client.http.impl.HttpClient.createStub(HttpClient.java:89)
        at com.vmware.evo.sddc.common.client.vmware.BaseVmomiClient.createStub(BaseVmomiClient.java:91)
        at com.vmware.evo.sddc.common.client.vmware.vsphere.VcManagerBase.listAvailableVmnicsForHost(VcManagerBase.java:12372)
        at com.vmware.vcf.common.fsm.plugins.action.impl.ManagementPortgroupUplinkTeamingValidationAction.hostAvailableVmnic(ManagementPortgroupUplinkTeamingValidationAction.java:189)
        at com.vmware.vcf.common.fsm.plugins.action.impl.ManagementPortgroupUplinkTeamingValidationAction.execute(ManagementPortgroupUplinkTeamingValidationAction.java:90)
        at com.vmware.vcf.common.fsm.plugins.action.impl.ManagementPortgroupUplinkTeamingValidationAction.execute(ManagementPortgroupUplinkTeamingValidationAction.java:43)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)

We also observe that the host will go to the following error state:

platform=# select id,hostname,name,status from host where status='ERROR';
                  id                  |          hostname           | name | status

--------------------------------------+-----------------------------+------+--------

 7bd8d87a-3841-xxxx-81fb-fcb1489xxxx |Hostname.com |    Hostname  | ERROR

Environment

Vmware Cloud Foundation 4.x

Cause

This issue is caused due to a cross-product gap when doing an nslookup for the host.

Resolution

To workaround the issue, please follow the steps mentioned below:

This validation logic can be skipped by adding the following application property validation.disable.mgmt.portgroup.uplink.teaming.check in the domain manager service:

SSH to the SDDC Virtual Machine and open /etc/vmware/vcf/domainmanager/application.properties.
Add validation.disable.mgmt.portgroup.uplink.teaming.check=true in a new line.
Restart the services: /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
Retry the operation.

This procedure should be performed once, and it will be applicable for all subsequent running operations.

The customers should manually verify that the host has at least one free VMNIC available, which is going to be attached to one of the uplinks connected to the management port group.