Containers Unable to Modify/See File Permissions Within NFS Share

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

• Containers are able to mount an NFSv4 Share, but are unable to modify/see file permissions within the share as they are displayed as nobody:nogroup
• /etc/idmapd.conf file on the worker nodes must be changed to add the following line: Domain = <domainname>
• Share is mounted using an NFS Claim Provider. 

Environment

• VMware Tanzu Kubernetes Grid Integrated Edition

Resolution

Leverage os-conf and the pre-start-script job to make changes to the OS during VM creation.

Upload os-conf-release

bosh upload-release --sha1 daf34e35f1ac678ba05db3496c4226064b99b3e4 \
"https://bosh.io/d/github.com/cloudfoundry/os-conf-release?v=22.2.1"

 

Confirm upload

bosh releases | grep os-conf
os-conf                   22.2.1                     a2154d6

 

Create runtime config

The pre-start script runs in parallel to os-conf and operates on the the same file /etc/idmapd.conf for kubernetes dependancies, which can cause a race condition. Adding a sleep loop to check that the file is in the correct state before appending is necessary. Make sure to include or exclude deployments and/or instance_groups based on your business requirements:

cat <<EOFA > runtime.yaml
releases: 
- name: "os-conf"
  version: "22.2.1" 
addons:
- name: os-configuration
  exclude: 
    deployments: [pivotal-container-service-<guid>]  
  include:
    instance_groups: [worker]
  jobs:
  - name: pre-start-script
    release: os-conf
    properties:
      script: |-
        #!/bin/bash
        while ! grep -q "Mapping" /etc/idmapd.conf; do sleep 1; done; echo "Domain = my.domain.com" >> /etc/idmapd.conf
EOFA

Update bosh runtime config

bosh update-runtime-config runtime.yaml

Verify bosh runtime config

bosh runtime-config

---
addons:
- exclude:
    deployments:
    - pivotal-container-service-<guid>
  include:
    instance_groups:
    - worker
  jobs:
  - name: pre-start-script
    properties:
      script: |-
        #!/bin/bash
        while ! grep -q "Mapping" /etc/idmapd.conf; do sleep 1; done; echo "Domain = my.domain.com" >> /etc/idmapd.conf
    release: os-conf
  name: os-configuration
releases:
- name: os-conf
  version: 22.2.1

Upgrade cluster

A cluster upgrade in required for the changes to take.

tkgi upgrade-cluster <cluster>

Additional Information

• Bosh os-conf/22.3.0 documentation