Tanzu Kubernetes Grid Integrated Management Console apply configuration fails and reduces Bosh Director disk size

Products

• VMware Tanzu Kubernetes Grid Integrated Edition
• VMware Tanzu Kubernetes Grid Integrated Edition 1.x
• VMware Tanzu Kubernetes Grid Integrated Edition (Core)

Issue/Introduction

• Apply configuration from Tanzu Kubernetes Grid Integrated Management Console(TKGIMC) will fail and attempt to reduce the bosh director persistent disk.
• Validation of the Bosh Director VM disk usage with df -h shows /var/vcap/store at 100%: 
  /dev/sdd1       493G  468G     0 100% /var/vcap/store
    

Environment

• Tanzu Kubernetes Grid Integrated and Ops Manager deployed by Tanzu Kubernetes Grid Integrated Management Console(TKGIMC).
• At some point the Bosh Director disk was edited outside of the management of TKGIMC.

Cause

• The bosh director persistent disk size was previously increased on the Ops Manager side but not the TKGIMC appliance. This causes a mismatch between Ops Manager  and TKGIMC appliance manifests.
• When the Apply Configuration change gets applied from TKGIMC the disk size is reduced than what it is currently causing a failure.

Resolution

• Notes:

  • Modifying the bosh director settings needs to be done on the TKGIMC side and not Ops Manager.
  • The size needs to be selected based on the "Persistent Disk Type" available from the drop-down menus on the BOSH Director Tile >> Resource Config >> Under Persistent Disk Type.

 

• • After modifying the "bosh_persistent_disk_type"  value, the apply configuration from the TKGIMC side will recreate the bosh director VM with the new disk size even if the change of the size was already  done on the Ops Manager.

 

1. Open a browser using the TKGIMC appliance IP address or FQDN and login using the root account.
2. Expand "Configuration" and go to the "TKGI Configuration" view of the management console.
3. Select the "Wizard" option to be taken to the configuration wizard for Tanzu Kubernetes Grid Integrated Edition.
4. Click on  "Generate Configuration".
5. Then the TKGI Configuration YAML Editor page will appear and you can add the YAML data.
6. Ensure before  modifying the "bosh_persistent_disk_type" setting to backup the current configuration by clicking on "Export Yaml" button.
7. Using the Editor, locate  the "bosh_persistent_disk_type" key.
8. Modify the "bosh_persistent_disk_type" value to the  size you need based on the available disk sizes from the drop-down menus on the BOSH Director Tile >> Resource Config >> Under Persistent Disk Type.This must be in megabytes. 
   
   • 
9. After modifying bosh_persistent_disk_type  Click on Apply Configuration.
10. The Apply Configuration will start a task to create the bosh director VM with a new persistent disk size.