Tanzu Kubernetes Grid Integrated Management Console apply configuration fails and reduces Bosh Director disk size

Products

• VMware Tanzu Kubernetes Grid Integrated Edition
• VMware Tanzu Kubernetes Grid Integrated Edition 1.x
• VMware Tanzu Kubernetes Grid Integrated Edition (Core)

Issue/Introduction

• Apply configuration from Tanzu Kubernetes Grid Integrated Management Console(TKGIMC) will fail and attempt to reduce the bosh director persistent disk.
• Validate the Bosh Director VM disk usage is full by running df -h and verifying /var/vcap/store reports it is at 100%: 
  /dev/sdd1       493G  468G     0 100% /var/vcap/store
    

Environment

• Tanzu Kubernetes Grid Integrated and Ops Manager deployed by Tanzu Kubernetes Grid Integrated Management Console(TKGIMC).
• At some point the Bosh Director disk was edited outside of the management of TKGIMC.

Cause

• The Bosh Director persistent disk size was previously increased on the Ops Manager side but not the TKGIMC appliance. This causes a mismatch between Ops Manager and TKGIMC appliance manifests.
• When the Apply Configuration change gets applied from TKGIMC the disk size gets reduced from its current size and causes a failure.

Resolution

• Notes:

  • Modifying the Bosh Director settings needs to be done on the TKGIMC side and not Ops Manager.
  • The size needs to be selected based on the "Persistent Disk Type" available from the drop-down menus on the Bosh Director Tile >> Resource Config >> Under Persistent Disk Type.

 

• • After modifying the "bosh_persistent_disk_type"  value, the apply configuration from the TKGIMC side will recreate the Bosh Director VM with the new disk size even if the change of the size was already done on the Ops Manager.

 

1. Open a browser using the TKGIMC appliance IP address or FQDN and login using the root account.
2. Expand "Configuration" and go to the "TKGI Configuration" view of the management console.
3. Select the "Wizard" option to be taken to the configuration wizard for Tanzu Kubernetes Grid Integrated Edition.
4. Click on "Generate Configuration".
5. Then the TKGI Configuration YAML Editor page will appear and you can add the YAML data.
6. Ensure before modifying the "bosh_persistent_disk_type" setting to backup the current configuration by clicking on "Export Yaml" button.
7. Using the Editor, locate  the "bosh_persistent_disk_type" key.
8. Modify the "bosh_persistent_disk_type" value to the size you need based on the available disk sizes from the drop-down menus on the BOSH Director Tile >> Resource Config >> Under Persistent Disk Type. This must be set in megabytes. 
   
   • 
9. After modifying bosh_persistent_disk_type click on Apply Configuration.
10. The Apply Configuration will start a task to create the Bosh Director VM with a new persistent disk size.