Remove hosts from cluster fails - Cannot read the array length because "<local4>" is null

Products

• VMware SDDC Manager

Issue/Introduction

Remove ESXi hosts from a cluster fails in the SDDC manager. These hosts were forcefully removed from the NSX manager during the process.

The task fails at - Remove vmknic(s) from ESXi Hosts.
Error Message -  Cannot read the array length because "<local4>" is null 

Environment

VMware Cloud Foundation 5.x

Cause

SDDC is unable to find the vmknic entry as the hosts has encountered manual changes from the NSX manager. SDDC returns a null value when trying to remove the vmknic. This can be confirmed by finding the error line Cannot read the array length because "<local4>" is null

domainmanager.log: 

2025-02-14T08:45:01.452+0000 DEBUG [vcf_dm,67af028c41c100e1ae1e9060da695e3f,ba28] [c.v.v.c.f.p.a.i.RemoveVmknicsAction,dm-exec-11]  Removing vxlan Vmknic-s from host host_fqdn
2025-02-14T08:45:01.452+0000 INFO  [vcf_dm,67af028c41c100e1ae1e9060da695e3f,ba28] [c.v.e.s.c.c.v.vsphere.VcManagerBase,dm-exec-11]  
Removing vmknics with netstack keys [vxlan] from host https://host_fqdn/sdk.
 
2025-02-14T08:45:01.468+0000 ERROR [vcf_dm,67af028c41c100e1ae1e9060da695e3f,ba28] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-11]  [6QJ01C] VSPHERE_REMOVE_VMKNICS_FAILED Failed to remove vmknics from host host_fqdn
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to remove vmknics from host host_fqdn
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:104)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:36)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
    
Caused by: java.lang.NullPointerException: Cannot read the array length because "<local4>" is null
        at com.vmware.evo.sddc.common.client.vmware.vsphere.VcManagerBase.removeVmknicsByNetStackInstanceKey(VcManagerBase.java:13410)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.removeAllVmnics(RemoveVmknicsAction.java:131)
        at com.vmware.vcf.common.fsm.plugins.action.impl.RemoveVmknicsAction.execute(RemoveVmknicsAction.java:84)
        ... 16 common frames omitted

Resolution

Once the SDDC manager has initiated the remove hosts from cluster, we should not make any changes on these hosts from the NSX-T manager or the vCenter server.

If changes have been performed then SDDC is unaware of the change, which will eventually fail to complete the task. If something has been manually changed, then we will need to update the SDDC manager workflow that this change has been made. The only way to achieve this is to skip the particular workflow from the SDDC DB.
Kindly contact GSS for the assistance.