SMB with Linux diego_cell - Stale file handle

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

We had seen a large deployment in which upgrading from xenial to jammy stemcells caused all apps using SMB mounts to fail with ‘Stale file handle’ and permission denied errors. E.g:

 

vcap@aabbccddd-1234-9876-0000-2222:/var/mytest$ cat myfile.txt cat: myfile.txt: Stale file handle

 

2023-02-10T09:58:08.81-0300 [APP/TASK/123456789-1234-4321-3333-4444444444-|-abcdef12-9876-abcd-1111-4839202010101/0] OUT Access to the path '/var/vcap/path/to/file' is denied.

 

Environment

Upgrading from xenial to jammy stemcell

Cause

The stale file handle error seems to occur when a file inode reference is removed and no longer accessible. 

This turned out to be because the SMB server was suggesting inode numbers, instead of allowing the client to generate temporary inode numbers. 

Resolution

TAS 3.07 and 2.12.24, 2.11.35, 2.13.15 and TAS 4+ allow the"noserverino" mount parameter which forces the client to generate inode numbers itself rather than using the actual ones from the server. Passing this parameter needs re-binding of the SMB service for each application.

They also introduce a new checkbox in TAS -> App Containers to force noserverino in SMB mount across the whole foundation, rather than
relying on application authors to re-bind their SMB services.

Additional Information

https://github.com/cloudfoundry/smb-volume-release/commit/9f36ee30f1dc99a53b6bd66db9c0caaaf84aae55