Failed to mount an NFS datastore on an ESXi host with the error: "Unable to connect to NFS server: VSI node (####:)".
search cancel

Failed to mount an NFS datastore on an ESXi host with the error: "Unable to connect to NFS server: VSI node (####:)".

book

Article ID: 385243

calendar_today

Updated On:

Products

VMware vSphere ESXi VMware vCenter Server

Issue/Introduction

  • Unable to mount NFS datastores from vCenter GUI 
  • In the firewall ruleset, you will notice that the rules for nfs41client is missing when compared to working hosts where the NFS datastore is mounted
  • Manually mounting the NFS share from ESXi fails with an error. 
    An error occurred during host configuration :. Operation failed, diagnostics report: Mount failed: Unable to complete Sysinfo operation. Please see the VMkernel log file for more details .: Unable to connect to NFS server: VSI node (####)

     

  • Error in /var/run/log/vmkernel.log in ESXi host 
    In(182) vmkernel: cpu4:2####1)NFS: 2861: Retrying mount request for NFS_ESXi Server: 10.#.#.# (10.#.#.#) Volume: /NFS_ESXi
In(182) vmkernel: cpu4:2####1)NFS: 2861: Retrying mount request for NFS_ESXi Server: 10.#.#.# (10.#.#.#) Volume: /NFS_ESXi
In(182) vmkernel: cpu3:2####1)NFS: 2861: Retrying mount request for NFS_ESXi Server: 10.#.#.# (10.#.#.#) Volume: /NFS_ESXi
In(182) vmkernel: cpu20:2####1)NFS: 2861: Retrying mount request for NFS_ESXi Server: 10.#.#.# (10.#.#.#) Volume: /NFS_ESXi
    In(182) vmkernel: cpu46:2#####2 opID=8#####f)NFS: 349: Command: (mount) Server: (10.##.##.##) IP: (10.##.##.##) Connections: (1) Vmknic: (None)  Path: (<nfs datastore path>) Label: (<NFS Datastore name>) Options: (None)
In(182) vmkernel: cpu46:2#####2 opID=8#####f)NFS: 3500: NFS mount failed for 10.##.##.##:<nfs datastore path> volume <NFS Datastore name>. Status: Unable to connect to NFS server.
In(182) vmkernel: cpu35:2098769 opID=e#####5)NFS: 349: Command: (mount) Server: (10.##.##.##) IP: (10.##.##.##) Connections: (1) Vmknic: (None)  Path: (<nfs datastore path>) Label: (<NFS Datastore name>) Options: (None)



  • Error in /var/run/log/hostd.log in ESXi host 
    Er(163) Hostd[2#####7]: [Originator@6876 sub=Hostsvc.FSVolumeProvider opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] VmkCtl mount or ProcessNas failed: Mount failed: Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.: Unable to connect to NFS server: VSI node (####:)
In(166) Hostd[2#####6]: [Originator@6876 sub=Hostsvc.VmkVprobSource] VmkVprobSource::Post event: (vim.event.EventEx) {
In(166) Hostd[2#####7]: -->    key = 252,
In(166) Hostd[2#####7]: -->    chainId = 1,
In(166) Hostd[2#####7]: -->    createdTime = "YYYY-MM-DDTHH:MN:SSZ",
In(166) Hostd[2#####7]: -->    userName = "",
In(166) Hostd[2#####7]: -->    host = (vim.event.HostEventArgument) {
In(166) Hostd[2#####7]: -->       name = "<hostname>",
In(166) Hostd[2#####7]: -->       host = 'vim.HostSystem:ha-host'
In(166) Hostd[2#####7]: -->    },
In(166) Hostd[2#####7]: -->    eventTypeId = "esx.problem.vmfs.nfs.mount.failed",
In(166) Hostd[2#####7]: -->    arguments = (vmodl.KeyAnyValue) [
In(166) Hostd[2#####7]: -->       (vmodl.KeyAnyValue) {
In(166) Hostd[2#####7]: -->          key = "1",
In(166) Hostd[2#####7]: -->          value = "10.##.##.##"
In(166) Hostd[2#####7]: -->       },
In(166) Hostd[2#####7]: -->       (vmodl.KeyAnyValue) {
In(166) Hostd[2#####7]: -->          key = "2",
In(166) Hostd[2#####7]: -->          value = "<NFS Datastore Path>"
In(166) Hostd[2#####7]: -->       },
In(166) Hostd[2#####7]: -->       (vmodl.KeyAnyValue) {
In(166) Hostd[2#####7]: -->          key = "3",
In(166) Hostd[2#####7]: -->          value = "<NFS Datastore Name>"
In(166) Hostd[2#####7]: -->       },
In(166) Hostd[2#####7]: -->       (vmodl.KeyAnyValue) {
In(166) Hostd[2#####7]: -->          key = "4",
In(166) Hostd[2#####7]: -->          value = "Unable to connect to NFS server."
In(166) Hostd[2#####7]: -->       }
In(166) Hostd[2#####7]: -->    ],
In(166) Hostd[2#####7]: -->    objectId = "ha-host",
In(166) Hostd[2#####7]: -->    objectType = "vim.HostSystem",
In(166) Hostd[2#####7]: --> }
In(166) Hostd[2#####6]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 217 : NFS mount failed for 10.##.##.##:<NFS Datastore Path> volume <NFS Datastore Name>. Status: Unable to connect to NFS server.
In(166) Hostd[2#####7]: [Originator@6876 sub=AdapterServer opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] AdapterServer caught exception; <<5#####8-####-####-####-######e1, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 31565'>>, ha-datastoresystem, vim.host.DatastoreSystem.createNasDatastore, <vim.version.v8_0_3_0, internal, 8.0.3.0>, [N11HostdCommon18VmomiAdapterServer19ActivationResponderE:0x000000fca5f3a688]>, N7Hostsvc21HaPlatformConfigFault9ExceptionE(Fault cause: vim.fault.PlatformConfigFault
In(166) Hostd[2#####7]: --> )
In(166) Hostd[2#####7]: --> [context]zK##################################AA==[/context]
In(166) Hostd[2#####7]: [Originator@6876 sub=Vimsvc.TaskManager opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] Task Completed : haTask-ha-host-vim.host.DatastoreSystem.createNasDatastore-1267 Status error
In(166) Hostd[2#####7]: [Originator@6876 sub=Solo.Vmomi opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] Activation finished; <<5#####8-####-####-####-######e1, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : 31565'>>, ha-datastoresystem, vim.host.DatastoreSystem.createNasDatastore, <vim.version.v8_0_3_0, internal,
n18VmomiAdapterServer19ActivationResponderE:0x000000fca5f3a688]>
Db(167) Hostd[2#####7]: [Originator@6876 sub=Solo.Vmomi opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] Arg spec:
Db(167) Hostd[2#####7]: --> (vim.host.NasVolume.Specification) {
Db(167) Hostd[2#####7]: -->    remoteHost = "10.##.##.##",
Db(167) Hostd[2#####7]: -->    remotePath = "<NFS Datastore Path>",
Db(167) Hostd[2#####7]: -->    localPath = "<NFS Datastore Name>",
Db(167) Hostd[2#####7]: -->    accessMode = "readWrite",
Db(167) Hostd[2#####7]: -->    type = "NFS",
Db(167) Hostd[2#####7]: -->    remoteHostNames = (string) [
Db(167) Hostd[2#####7]: -->       "10.##.##.##"
Db(167) Hostd[2#####7]: -->    ],
Db(167) Hostd[2#####7]: -->    vmknicBound = false,
Db(167) Hostd[2#####7]: --> }
In(166) Hostd[2#####7]: [Originator@6876 sub=Solo.Vmomi opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] Throw vim.fault.PlatformConfigFault
In(166) Hostd[2#####7]: [Originator@6876 sub=Solo.Vmomi opID=mf####-####42-auto-ak4t5-h5:71######-##-##-e859 sid=5#####8 user=vpxuser:<DOMAIN\USER>] Result:
In(166) Hostd[2#####7]: --> (vim.fault.PlatformConfigFault) {
In(166) Hostd[2#####7]: -->    faultMessage = (vmodl.LocalizableMessage) [
In(166) Hostd[2#####7]: -->       (vmodl.LocalizableMessage) {
In(166) Hostd[2#####7]: -->          key = "com.vmware.esx.hostctl.default",
In(166) Hostd[2#####7]: -->          arg = (vmodl.KeyAnyValue) [
In(166) Hostd[2#####7]: -->             (vmodl.KeyAnyValue) {
In(166) Hostd[2#####7]: -->                key = "reason",
In(166) Hostd[2#####7]: -->                value = "Mount failed: Unable to complete Sysinfo operation.  Please see the VMkernel log file for more details.: Unable to connect to NFS server: VSI node (####:) "
In(166) Hostd[2#####7]: -->             }
In(166) Hostd[2#####7]: -->          ],
In(166) Hostd[2#####7]: -->       }
In(166) Hostd[2#####7]: -->    ],
In(166) Hostd[2#####7]: -->    text = "",
In(166) Hostd[2#####7]: -->    msg = ""
In(166) Hostd[2#####7]: --> }

Environment

VMware vCenter Server 
VMware vSphere ESXi 

Cause

This issue is observed because the VLAN used for the NFS connectivity is not allowed on the switch port interface

Cause Validation

For example, if the NFS VMkernel interface is configured with VLAN 741 and the physical adapter (vmnic) used for the NFS connection is set to handle VLAN 741 traffic, you can confirm this by navigating to:
Host > Configure > Networking > Physical Adapters > Observed IP Ranges.

However, upon reviewing the switch configuration, you will find that VLAN 741 is not allowed on the relevant switch port.

Note: To verify and correct this, coordination with your physical networking team is necessary, as this information is not available within vCenter 

Resolution

Engage your physical networking team to review the configuration of the switch port connected to the ESXi host. Ensure that the VLAN used for the NFS traffic is explicitly allowed on that interface.

Once the VLAN is permitted on the switch port, the NFS vmkernel interface should be able to communicate with the NFS datastore successfully.

Powered by Wolken Software