Workload Cluster Nodes With Multiple Ephemeral Volume Mounts Are Swapped After Reboot

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• This issue is seen when multiple ephemeral volume mounts are used on workload cluster worker nodes with disks of differing sizes. When the node is rebooted the sizes of the disks will swap. Below is an example of how the YAML file looks:
  
  volumes:
    - name: containerd
      mountPath: /var/lib/containerd
      capacity:
        storage: 100Gi
    - name: kubelet
      mountPath: /var/lib/kubelet
      capacity:
        storage: 200Gi
• When accessing the node via an SSH connection and running the df -h command the below is observed:
  
  /dev/sdb1 20G 7.2G 200G 15% /var/lib/containerd
  /dev/sdc1 157G 880K 100G 10% /var/lib/kubelet
• This issue occurs on vSphere with Tanzu environments built on vSphere 7.0 U3J and earlier builds as well as vSphere 8.0 builds prior to U1
  
  
  

Environment

• VMware vCenter Server 7.0.x
• VMware vCenter Server 8.0.x

Cause

This is a known issue identified by VMware engineering.

Resolution

This is confirmed by engineering to be resolved in following releases of vCenter Server

Release Name	Version	Release Date	Build Number
vCenter Server 8.0 Update 1	8.0.1.00000	2023-04-18	21560480
vCenter Server 7.0 Update 3k	7.0.3.01300	2023-02-23	21290409

Workaround:

When provisioning worker nodes/node pools with multiple ephemeral volume mounts use the same size disk for each ephemeral volume mount. This is confirmed by engineering to be resolved in following releases of vCenter Server.