Loss of Azure Tags in Cloud Zones at the Availability Zone Level

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Loss of tags under Azure for Availability Zone objects at the Cloud Zone level.

1. It seems to happen randomly at any time.
2. The loss of tags is limited to the Azure account.
3. It is not always the same Cloud Zone that has the problem.

Environment

Aria Automation 8.16.2

Cause

 

• Azure has some regions where they support different counts of zones, and some regions where no zones are supported at all. So, we are enumerating the zones from the cloud. If the enumeration fails, there is a scheduled mechanism that cleans resources that are older than 1-2 enumeration cycles. Basically, they get removed as we keep vRA data in sync with the cloud. And if the cloud doesn't return a zone, it may be not supported anymore, therefore removing it.

• We can see in the logs that sometimes the enumeration of cloud zones fails, which leads to the zone being removed and recreated on the next enumeration cycle, therefore the "missing" tags.

  • 25-03-04T19:06:55.709Z WARN public-adapters [host='adapter-host-service-app-549ccd568-j7t4m' thread='azure-executors-29' user='' org='' trace='' parent='' span=''] c.v.p.c.m.a.a.e.AzureComputeEnumerationAdapterService.lambda$collectAvailabilityZones$6:738 - [/resources/endpoints/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx / [canadacentral]] Error occurred while collecting availability zones from cloud.

Resolution

 

Steps to add configuration to the provisioning-service-app deployment:

SSH to Aria Automation appliance
1. Run command to edit provisioning-service-app deployment.

kubectl -n prelude edit deployment provisioning-service-app

2. Add following line under "JAVA_OPTS" section

-Dphoton-model.image.enumeration.azure.filter=nopub:nooffer:nosku:

-Dphoton-model.EvictionRestClientCache.expireAfterSeconds=3600

Note: For 3 nodes VRA edit the deployment on any of the VRA node (only on one) 

 

3. Restart the provisioning pods to apply the changes 

kubectl -n prelude rollout restart deployment provisioning-service-app

Note: that this approach will require brief downtime as it will interrupt any activities currently running in the provisioning services pods.

 

4. Validate that the properties were applied 

kubectl -n prelude describe deployment  provisioning-service-app | grep  Dphoton

You can check it on all VRA nodes to be sure that the settings were applied

5. Monitor the tags presence on affected computes.