Failed reading persistent disk settings while upgrading Bosh vSphere stemcell

Products

• Operations Manager

Issue/Introduction

When performing a Bosh vSphere stemcell upgrade, Bosh CPI hits an issue while updating a VM instance that has got a persistent disk
From the Apply Change logs:

Task 7712 | 08:11:00 | Updating instance backup_restore: backup_restore/03b614a9-9658-4654-abbb-12c78cadabb1 (0) (canary) (00:02:05)
                       L Error: Action Failed get_task: Task ab365544-082d-46f1-4e84-3c14cefc321c result: Reading persistent disk settings: Persistent disk with volume id 'disk-5c5f56c3-7d61-4d4a-abbb-fad69d5d0321' could not be found

Environment

Product Version: 2.1

Resolution

There is a known issue with vSphere stemcell version 621.151 and version 456.186 

Known issues

• We’ve seen failures with this version of the stemcell in vSphere when attempting to attach a persistent disk to a running VM. We are currently planning to address this issue with an update to the vSphere CPI. It should be fixed in vSphere CPI release >= v69

To resolve this known issue, please use the OpsManager version 2.10.17 which includes vSphere CPI release 69, and the vSphere stemcell version 621.154 / 456.188 for any tiles (TAS, TKGi ...)

2.10 Release Notes: https://docs.vmware.com/en/VMware-Tanzu-Operations-Manager/2.10/vmware-tanzu-ops-manager/release-notes.html

Downgrade stemcell workaround (not recommended):
If you cannot use the permanent resolution above, you may revert the stemcell by following this KB https://knowledge.broadcom.com/external/article?articleNumber=298731 


Please contact Tanzu Support if any assistance is needed following the KB. Please note that a single typo in the installation.yml file may cause OpsManager GUI to crash. 

Modifying installation and product template files: https://docs.vmware.com/en/VMware-Tanzu-Operations-Manager/2.10/vmware-tanzu-ops-manager/install-modify-ops-man.html