Explicit network tagging suddenly breaks down during provisioning requests leaving deployments stuck in progress

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• You use capability tags on your networks as your primary tagging strategy
  • You then consume those capability tags as constraints during blueprint provisioning.
• During these provisioning requests, deployments experience intermittent timeouts and inconsistent provisioning completion, for instance, 6 deployments with 2 virtual machines each only have an average 50% success rate.
• When reviewing provisioning diagrams, you see a large network of red lines which represent a large number of individual filtering checks against the networks during placement.
• This issue appears similar to that of slowness in the VAPI in vSphere, but no other blueprints necessarily experience these symptoms and Aria Automation provisions these systems without issue.

Environment

VMware Aria Automation 8.18.x

Cause

When scaling up an environment and only using network tagging without Cloud Zone or Compute clustering filtering, the reservation calculations timeout and cause OOM errors within Aria Automation causing reservation filtering to fail and leave deployments in an inconsistent state, ultimately failing the request.

Resolution

In addition to existing network tags, capability tag your compute or Cloud Zones and use constraints inside your blueprint to filter down to the appropriate cluster resource. This eliminates the large number of requests performed by the system when solely checking against networks across all your configured cloud accounts.