vSAN FSVM Mount Failure – “No such file or directory” Error Observed When Mounting File Share
search cancel

vSAN FSVM Mount Failure – “No such file or directory” Error Observed When Mounting File Share

book

Article ID: 405232

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

Symptoms

One or more NFS clients—such as Kubernetes pods or Linux virtual machines - may fail to mount a vSAN file share with the following error:

# mount containername:/vsanfs/Sharename /root/mnt/
mount.nfs: mounting containername:/vsanfs/Sharename failed, reason given by server: No such file or directory

Additional Ganesha/NFS Logs:

  • The stat() operation failed to resolve the export directory.

  • The Ganesha NFS daemon was unable to initialize the export.

  • NFS clients received the message: "Invalid export identifier 163".

  • Input/output (I/O) errors occurred during feature enablement operations, such as when executing chmod on the FSVM.

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

The issue occurs when the backing volume for a vSAN file share becomes inaccessible to a specific File Services Virtual Machine (FSVM). In the affected scenario, the export path cannot be resolved due to a volume mount failure within the FSVM.

Contributing factors include:

  • A stale or failed mount on the FSVM residing on the affected ESXi host.

  • The Ganesha NFS daemon fails to perform a stat() operation on the export directory.

  • The export path returns an Input/output error, preventing the share from being initialized.

Log Evidence - 

Log files located at: /scratch/log/vdfs_support/containers/fs_vm_logs/fsvm_logs/containers/<ContainerName>

Path Resolution Failure: These logs indicate that the FSVM was unable to resolve the export path due to a stat failure, resulting in a critical input/output error from the Ganesha daemon.

2025-06-14T09:09:43Z : epoch 684d3c53 : Container : ganesha.nfsd-137[none] [main] 1653 :claim_posix_filesystems :FSAL :Could not stat directory for path /vsfs/411d7c65-####-####-####-5cba2c84f600/volumes/dcf7444b-####-####-####-c863b95f6249/default
2025-06-14T09:09:43Z : epoch 684d3c53 : Container : ganesha.nfsd-137[none] [main] 240 :vdfs_create_export :FSAL :resolve_posix_filesystem(/vsfs/411d7c65-####-####-####-5cba2c84f600/volumes/dcf7444b-####-####-####-c863b95f6249/default) returned Input/output error (5)

nfs4_uid_to_name failed with code -2 - This entry shows that the NFS server encountered ID mapper lookup failures. While this is a secondary issue, it reflects that the export was not mounted successfully.

2025-06-14T09:09:46Z : epoch 684d3c53 : Container : ganesha.nfsd-137[::ffff:10.###.##.30] [svc_7] 234 :xdr_encode_nfs4_princ :ID MAPPER :nfs4_uid_to_name failed with code -2.

I/O Error During Feature Enablement - The FSVM failed to enable the treedelete feature due to an I/O error while attempting to modify file system attributes, further confirming that the volume was not accessible.

2025-07-13T02:26:30.364Z info [EndpointMonitor-4] [_ExecRun]  Output: chmod: cannot access '/vsfs/411d7c65-####-####-####-5cba2c84f600/volumes/dcf7444b-####-####-####-c863b95f6249/default/.vdfs/unlinked': Input/output error
2025-07-13T02:26:30.365Z error [EndpointMonitor-4] [EnableFeatureForShare] UPG: failed to enable treedelete for share SDP1154711-NFS-CRM-INDO Traceback (most recent call last):   File "/usr/lib/vmware/vsan/perfsvc/VDFSEndpointProtoUtil.py", line 1267, in EnableFeatureForShare   File "/usr/lib/vmware/vsan/perfsvc/VDFSEndpointContainerDocker.py", line 369, in Chmod VDFSEndpointContainer.ContainerError: Error in doing chmod for /vsfs/411d7c65-####-####-####-5cba2c84f600/volumes/dcf7444b-####-####-####-c863b95f6249/default/.vdfs/unlinked

Invalid Export ID Error - Clients attempting to mount the file share received responses indicating an invalid export ID, as the export never successfully initialized due to the failed path resolution on the FSVM.

2025-07-14T04:02:50Z : epoch 684d3c53 : Container : ganesha.nfsd-137[::ffff:10.###.##.30] [svc_3] 159 :nfs4_mds_putfh :DISP :NFS4 Request from client (::ffff:10.###.##.30) has invalid export identifier 163

Resolution

Workaround

  • Identify the FSVM Hosting the Share
    • Use the vSAN File Services UI to determine which FSVM hosts the impacted share.

  • Connect to FSVM console 
    • Run the following commands to investigate:

docker ps                  # Identify container ID
docker exec -it <containerID> bash
showmount -e localhost     # Verify export status

  • Reinitialize the Ganesha Service via Host Maintenance Mode
    • Place the affected ESXi host (where the FSVM is running) into Maintenance Mode using the Ensure Accessibility.

  • Exit Maintenance Mode
    • Upon exiting, the FSVM reinitializes the Ganesha service.
    • The previously inaccessible export path is reclaimed and initialized successfully.

Post-recovery, the vSAN file share mounts successfully from both application and pod levels. File and directory structures are listed without errors, confirming full functionality and access to the export path.

Powered by Wolken Software