Tanzu Application Service for Windows Stemcells deprecated on Azure Platform

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

If you are attempting to Apply Changes on Tanzu Application Service for Windows (TAS for Windows) Deployment and the Apply Changes fails during VM creation, it may be due to the Windows Stemcell version you are using.

Azure has marked certain Windows Stemcells as deprecated in the Azure Marketplace. Windows Stemcells which are marked as deprecated are unavailable for download during VM creation. 

Note: This issue only affects TAS for Windows running a Azure platform, other IaaS providers are unaffected by this issue.

You can confirm if you are affected by this issue by running this command:

`bosh task 89998 --debug`  

Search for the following string in the debug output:

`image is not found` 

 

Example debug output:

D, [2022-01-06T15:19:37.078488 #7555 #1960] DEBUG -- [req_id cpi-693192]: list_platform_image_versions(centralus, pivotal, bosh-windows-server-2019, 2019-sku2)
I, [2022-01-06T15:19:37.078921 #7555 #1960] INFO -- [req_id cpi-693192]: http_get - trying to get https://management.azure.com/subscriptions/SUBSCRIPTION_GUID/providers/Microsoft.Compute/locations/centralus/publishers/pivotal/artifacttypes/vmimage/offers/bosh-windows-server-2019/skus/2019-sku2/versions?api-version=2018-04-01
D, [2022-01-06T15:19:37.079130 #7555 #1960] DEBUG -- [req_id cpi-693192]: http_get_response - 0: GET, x-ms-client-request-id: a363d982-4602-4572-85fc-94eaa6d5d0cf, URI: https://management.azure.com/subscriptions/SUBSCRIPTION_GUID/providers/Microsoft.Compute/locations/centralus/publishers/pivotal/artifacttypes/vmimage/offers/bosh-windows-server-2019/skus/2019-sku2/versions?api-version=2018-04-01
D, [2022-01-06T15:19:37.231951 #7555 #1960] DEBUG -- [req_id cpi-693192]: http_get_response - 200
x-ms-client-request-id: a363d982-4602-4572-85fc-94eaa6d5d0cf
x-ms-request-id: 63b4d5d3-4296-4859-954e-6cadfcb27628
x-ms-correlation-request-id: 71c6ea7b-06d8-4901-a33d-19687761fd6b
x-ms-routing-request-id: CENTRALUS:20220106T151937Z:71c6ea7b-06d8-4901-a33d-19687761fd6b
response.body=> (Ruby hash) {
    "{\"location\"=>\"centralus\", \"name\"=>\"2019.27.026001\", \"id\"=>\"/Subscriptions/SUBSCRIPTION_GUID/Providers/Microsoft.Compute/Locations/centralus/Publishers/pivotal/ArtifactTypes/VMImage/Offers/bosh-windows-server-2019/Skus/2019-sku2/Versions/2019.27.026001\"}"=> null,
    "{\"location\"=>\"centralus\", \"name\"=>\"2019.32.006002\", \"id\"=>\"/Subscriptions/SUBSCRIPTION_GUID/Providers/Microsoft.Compute/Locations/centralus/Publishers/pivotal/ArtifactTypes/VMImage/Offers/bosh-windows-server-2019/Skus/2019-sku2/Versions/2019.32.006002\"}"=> null,
    "{\"location\"=>\"centralus\", \"name\"=>\"2019.42.027001\", \"id\"=>\"/Subscriptions/SUBSCRIPTION_GUID/Providers/Microsoft.Compute/Locations/centralus/Publishers/pivotal/ArtifactTypes/VMImage/Offers/bosh-windows-server-2019/Skus/2019-sku2/Versions/2019.42.027001\"}"=> null,
    "{\"location\"=>\"centralus\", \"name\"=>\"2019.43.062001\", \"id\"=>\"/Subscriptions/SUBSCRIPTION_GUID/Providers/Microsoft.Compute/Locations/centralus/Publishers/pivotal/ArtifactTypes/VMImage/Offers/bosh-windows-server-2019/Skus/2019-sku2/Versions/2019.43.062001\"}"=> null
}
D, [2022-01-06T15:19:37.232221 #7555 #1960] DEBUG -- [req_id cpi-693192]: list_platform_image_versions: The version '2019.41.030002' of the image is not found

[...]

E, [2022-01-06T15:19:53.663527 #7019] [create_missing_vm(compilation-9e3ccb93-2b06-4c76-b1f8-8190ba319d1b/9c650e77-8777-47a1-bd95-d3d908cea31c (0)/1)] ERROR -- DirectorJobRunner: failed to create VM, retrying (1)
E, [2022-01-06T15:19:53.663700 #7019] [create_missing_vm(compilation-9e3ccb93-2b06-4c76-b1f8-8190ba319d1b/9c650e77-8777-47a1-bd95-d3d908cea31c (0)/1)] ERROR -- DirectorJobRunner: error creating vm: CPI error 'Bosh::Clouds::VMCreationFailed' with message '#<Bosh::Clouds::VMCreationFailed: Given stemcell 'bosh-light-stemcell-8ef9bf5d-e1c0-4c19-8919-f88cc72cb635' does not exist> 

 

To confirm if the image has been deprecated, leverage the Azure CLI and search for the image using its URN:

$ az vm image terms show --urn "pivotal:bosh-windows-server-2019:2019-sku2:2019.40.034001"
(ImageVersionDeprecated) VM Image from publisher: pivotal with - Offer: bosh-windows-server-2019, Sku: 2019-sku2, Version: 2019.40.034001 is deprecated.
Code: ImageVersionDeprecated
Message: VM Image from publisher: pivotal with - Offer: bosh-windows-server-2019, Sku: 2019-sku2, Version: 2019.40.034001 is deprecated. 

 

Resolution

If you confirm the Windows Stemcell version you are attempting to use is marked as deprecated, then you will need to use one of the available versions in order to continue with your TAS for Windows activities.

To identify which Windows stemcell versions are available, run the following command:

$ curl -s -X GET -H "Authorization: Bearer $(az account get-access-token | jq -r .accessToken)" -H "Content-Type:application/json" -H "Accept:application/json" https://management.azure.com/subscriptions/$(az account list | jq -r '.[]?.id')/providers/Microsoft.Compute/locations/westeurope/publishers/pivotal/artifacttypes/vmimage/offers/bosh-windows-server-2019/skus/2019-sku2/versions?api-version=2018-04-01 | jq -r '.[]?.name'

2019.27.026001
2019.32.006002
2019.42.027001
2019.43.062001
2019.44.010001 

 

For Version 2019.41

As of 1/25 Version 2019.41.03002 has been uploaded to Tanzu Network and can be downloaded to replace the current 2019.41 stemcell on your platform.

To implement the newly downloaded stemcell utilize bosh to upload and repair the stemcell
 

bosh upload-stemcell light-bosh-stemcell-2019.41-azure-hyperv-windows2019-go_agent.tgz --fix