VM deployment for multiple VM's fail post upgrade of Aria Automation to 8.16.1 or later
search cancel

VM deployment for multiple VM's fail post upgrade of Aria Automation to 8.16.1 or later

book

Article ID: 371211

calendar_today

Updated On:

Products

VMware Aria Suite

Issue/Introduction

Symptoms

  • Multiple deployments are being provisioned with no more than 20 Windows Server virtual machines into vCenter.
  • ABX actions with PowerShell 7.2 runtime are ran for each VM and waits for VM Customization to succeed, then the VM enters a running state.
    • A script is then run in order to add users to the local Administrators group.
  • This process started consuming more memory after the upgrade from 8.14.0 to 8.16.1.
    • No improvements after upgrading to 8.16.2 or higher.
  • Increasing the memory per action alleviates the issue, but many actions continue to fail as there is no longer sufficient ABX service memory.

Environment

Aria Automation 8.16.1 or later

Cause

The polyglot base image used in newer releases now consume more memory on average.

Resolution

Prerequisites

  • Take simultaneous VM snapshots without memory of all 3 Aria Automation appliances.
    • This can be done with a Day 2 operation from Aria Suite Lifecycle.

Procedure: Implementation

  1. Run the following command on one Aria Automation node to create a custom profile for the cluster:
    vracli cluster exec -- bash -c 'base64 -d <<< IyEvYmluL2Jhc2gKCmg9JChob3N0bmFtZSkKbG9nKCkgewogIGVjaG8gIlskMV1bJChkYXRlICIrJVktJW0tJWQtJUgtJU0tJVMiKV1bJGhdICQyIiA+JjIKfQoKbG9nIElORk8gIkNyZWF0aW5nIHByb2ZpbGUgZGlyZWN0b3J5Igpta2RpciAtcCAvZXRjL3Ztd2FyZS1wcmVsdWRlL3Byb2ZpbGVzL2FieC1tZW1vcnktaW5jcmVhc2UtdnJhZS02Mjg3OC9oZWxtL3ByZWx1ZGVfYWJ4LXNlcnZpY2UvCmxvZyBJTkZPICJQcm9maWxlIGRpcmVjdG9yeSBjcmVhdGVkIgoKbG9nIElORk8gIkNyZWF0aW5nICJjaGVjayIgZmlsZSIKY2F0IDw8RU9GID4gL2V0Yy92bXdhcmUtcHJlbHVkZS9wcm9maWxlcy9hYngtbWVtb3J5LWluY3JlYXNlLXZyYWUtNjI4NzgvY2hlY2sKIyEvYmluL2Jhc2gKZXhpdCAwCkVPRgpjaG1vZCA3NTUgL2V0Yy92bXdhcmUtcHJlbHVkZS9wcm9maWxlcy9hYngtbWVtb3J5LWluY3JlYXNlLXZyYWUtNjI4NzgvY2hlY2sKbG9nIElORk8gIiJjaGVjayIgZmlsZSBjcmVhdGVkIgoKIyBFbnRlcmluZyBvbmx5IHByb3BlcnRpZXMgaW4gYWJ4LXNlcnZpY2UgdGhhdCBuZWVkIGNoYW5naW5nCmxvZyBJTkZPICJDcmVhdGluZyBwcm9maWxlIHJlc291cmNlcyBmaWxlIgpjYXQgPDxFT0YgPiAvZXRjL3Ztd2FyZS1wcmVsdWRlL3Byb2ZpbGVzL2FieC1tZW1vcnktaW5jcmVhc2UtdnJhZS02Mjg3OC9oZWxtL3ByZWx1ZGVfYWJ4LXNlcnZpY2UvOTAtcmVzb3VyY2VzLnlhbWwKYWJ4OgogIHByb3ZpZGVyUG9seWdsb3RSdW5uZXJNZW1vcnlMaW1pdDogMTFHCiAgcHJvdmlkZXJQb2x5Z2xvdFJ1bm5lck1lbW9yeVRocm90dGxpbmc6IDEwRwpFT0YKY2htb2QgNjQ0IC9ldGMvdm13YXJlLXByZWx1ZGUvcHJvZmlsZXMvYWJ4LW1lbW9yeS1pbmNyZWFzZS12cmFlLTYyODc4L2hlbG0vcHJlbHVkZV9hYngtc2VydmljZS85MC1yZXNvdXJjZXMueWFtbApsb2cgSU5GTyAiUHJvZmlsZSByZXNvdXJjZXMgZmlsZSBjcmVhdGVkIgo= | bash -' 
  2. Power off the Aria Automation cluster using Aria Suite Lifecycle.
  3. Login to vCenter and increase the memory allocated to each virtual machine in the cluster by 2GB.
  4. Power On the Aria Automation cluster using Aria Suite Lifecycle.
    Note: Performing this operation from Aria Suite Lifecycle will run deploy.sh automatically, starting Aria Automation services, and pick up the new memory profile for the ABX workers.

Procedure: Reversal

Revert these changes by following the below steps:

Optional: Power off the Aria Automation cluster using Aria Suite Lifecycle and reduce each virtual machines memory allocation by 2GB

  1. To remove the custom profile, SSH into one node in the Aria Automation cluster and run the following command:
    vracli cluster exec -- bash -c 'base64 -d <<< cm0gLXJmIC9ldGMvdm13YXJlLXByZWx1ZGUvcHJvZmlsZXMvYWJ4LW1lbW9yeS1pbmNyZWFzZS12cmFlLTYyODc4 | bash -'
  2. Restart services by running the following command:
    /opt/scripts/deploy.sh
Powered by Wolken Software