Error: MountVolume.SetUp failed for volume pvc-#### NodePublishVolume failed Kubernetes pods vSAN File Service NSX DFW

Products

• VMware vSAN

Issue/Introduction

Kubernetes pods fail to transition to a Running state and remain locked in ContainerCreating following a vSAN File Service upgrade. Storage volume mounts timing out prevent persistent volume (PV) attachments in VMware Cloud Foundation environments configured with NSX Distributed Firewall rules.

Symptoms:

• Pods remain stuck in ContainerCreating status indefinitely.

• kubectl describe pod <pod-name> returns the following errors:

MountVolume.SetUp failed for volume "pvc-####" : rpc error: code = Aborted desc = NodePublishVolume failed.

 rpc error: code = DeadlineExceeded desc = context deadline exceeded.

• NFS storage traffic between File Service Virtual Machines (FSVMs) and worker nodes is silently dropped.

 

Environment

VMware Cloud Foundation 5.x / 4.x

vSAN File Service OSA / ESA

VMware NSX (Distributed Firewall enabled)

Cause

During a vSAN File Service upgrade, the vSphere ESX Agent Manager (EAM) automatically deletes existing File Service Virtual Machines (FSVMs) and redeploys new appliances. This process generates new Virtual Machine objects with updated inventory names (for example, transitioning from vSAN File Service Node (3) to vSAN File Service Node (8)).

If NSX Distributed Firewall rules or Exclusion Groups rely on VM Name or Inventory Name matching criteria, the security policy breaks because the original VM object no longer exists. The DFW cannot resolve the network identity of the new FSVM, causing all NFS storage traffic to be blocked by the firewall's default deny rule.

Resolution

To resolve this issue and ensure connectivity persists across future lifecycle operations, update the NSX security configuration to use static network identifiers instead of dynamic inventory names.

Once connectivity is restored, the CSI driver will successfully complete the NodePublishVolume operation, and pods will transition to Running.

Additional Information

Understanding and Resetting Distributed Firewall (DFW) Rule Statistics in NSX-T

Add or Modify Distributed Firewall Rules