Unable to power on virtual machines on new ESXI host added to vSAN cluster
search cancel

Unable to power on virtual machines on new ESXI host added to vSAN cluster

book

Article ID: 433855

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms:

  • Newly added hosts in vSAN Cluster fail to allow creation of VMs.

    Error : Module VMMon power on failed. Failed to start the virtual machine. Failed to power on VM. Could not power on virtual machine: Failure. Failed to create swap file '/vmfs/volumes/vsan:522768e937d15a4f-9109b70e48abdbae/8c5dc55f-8e0e-d251-7d0f-040973cb6561/ADSPCDC01-4f9d3cd6.vswp' : Failure

  • vMotion to these newly added hosts fails as well.

  • The hosts are in maintenance mode to prevent VMs migration to them to avoid failed vmotion attempts.

  • These newly added hosts are Compute-only hosts (i.e. Does not contribute to storage - No disk-groups).

Environment

  • VMware vSAN 8.x

Cause

  • Issue is caused due to misconfiguration of incorrect vmkernel adapter configured on these newly added hosts to connect to Witness traffic. 

  • These new hosts fail to connect to Witness node over vmkX. Instead, they connect successfully over vmkY.

    [root@<esxi-host>:~ ] vmkping -I vmkX <Witness-node-IP>
    PING #.#.#.# (#.#.#.#): 56 data bytes
    sendto () failed (Network is unreachable)

    [root@<esxi-host>:~ ] vmkping -I vmkY <Witness-node-IP>
    PING #.#.#.# (#.#.#.#): 56 data bytes
    64 bytes from #.#.#.#: icmp_seq=0 tt1=57 time=47.673 ms
    64 bytes from #.#.#.#: icmp seq=1 ttl=57 time=48.199 ms
    64 bytes from #.#.#.#: icmp_seq=2 tt1=57 time=47.931 ms

    #.#.#.# ping statistics
    3 packets transmitted, 3 packets received, 0% packet loss
    round-trip min/avg/max = 47.673/47.934/48.199 ms

Resolution

Follow steps provided below:

  1. For all hosts, disable Witness traffic on vmkX and configure it on vmkY vmkernel adapter.
    (ESXi host > Configure > Networking > Vmkernel Adapter).

    vmkX : Through which newly added hosts fail to connect to Witness node.
    vmkY : Through which newly added hosts connect to witness node successfully.

  2. Remove all these hosts from maintenance mode.

  3. Test creating new VMs on these newly added hosts.