Snapshot based backups are not working on specific ESXi host

Products

• VMware vSphere ESXi

Issue/Introduction

• ESXi host is configured with multiple vmknics, e.g:
  
  • vmk0 > 10.0.10.1/24
  • vmk1 > 10.0.20.1/24
  • vmk2 -> 10.0.30.1/24
• A 3rd party backup tool is in 10.0.30.0/24 network.
• The backup tool is accessing the host through vmk0.
• When backups are run, the tool fails to take a backup. 
• Packet captures done on host's vmk0 show incomplete TCP 3-way handshake from the backup tool, e.g.:
  11:09:55.702678 IP 10.0.30.2.57341 > 10.0.10.1.902: Flags [SEW], seq 3714930389, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:09:58.701369 IP 10.0.30.2.57341 > 10.0.10.1.902: Flags [SEW], seq 3714930389, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
11:10:04.707719 IP 10.0.30.2.57341 > 10.0.10.1.902: Flags [S], seq 3714930389, win 8192, options [mss 1460,nop,nop,sackOK], length 0
• ESXi host's ARP table has (incomplete) entries for the backup tool's IP address:
  esxcli network ip neighbor list
  

  Output of : localcli network ip neighbor list -N defaultTcpipStack

  
  Neighbor        Mac Address        Vmknic  Expiry        State  Type
--------------  -----------------  ------  ------------  -----  ----
10.0.30.2       (incomplete)       vmk2         -15 sec         Invalid

Environment

VMware vSphere ESXi

Cause

• ESXi host is unable to route traffic internally between vmknics.
• (in this example) vmk2 should not be configured in the network of tool which is accessing ESXi hosts via vmk0. 

Resolution

This is expected behaviour (ESXi host is unable to internally route traffic between two vmknic interfaces). 

To resolve the issue, you can:

• Change the IP address of vmknic creating a conflict.
• Or alternatively, remove this vmknic. 

Additional Information

Also see:

• Configuring static routes for vmkernel ports on an ESXi host
• Multihoming on ESXi