The Enterprise PKS upgrade from 1.3.x to 1.4.0 fails.
When you run the bosh debug log command, you see output similar to the following:
1 of 8 pre-start scripts failed. Failed Jobs:pxc-mysql
From pxc-mysql pre-start logs, you see messages similar to the following:
panic: Cannot continue, insufficient disk space to complete migration
When you run the df command, you see that the /var/vcap/store file system is approaching 100% utilization.
You see several 1GB mysql replication logs under /var/vcap/store/mysql.
As a part of the Enterprise PKS upgrade, there is a mysql migration from MaridDB(Galera) to MySQL(Percona) which will make a backup for existing files. When /var/vcap/store doesn't have enough free space to handle this requirement, the Enterprise PKS upgrade will fail.
To work around this issue, Increase the size of the PKS VM from Opsmanager -> PKS tile -> Resource from the current configured size to a higher size to have some room for the migration backup.