Troubleshooting Aria Automation reconciliation of vMotion migrated VM's

Products

• VMware Aria Suite

Issue/Introduction

Starting in Aria Automation 8.18.1, reconciliation of managed VM's that are vMotioned is possible preventing the VM's from reporting a 'missing' status: https://techdocs.broadcom.com/us/en/vmware-cis/aria/aria-automation/8-18/vmware-aria-automation-release-notes.html

In some situations, a vMotion of a VM can still result in the machine going missing from Aria Automation inventory rendering the VM unavailable for management.

Environment

Aria Automation 8.18.1 and later

Cause

This can be caused by incorrect configuration of config mappings, failures leading to update status in Aria Automation database, and migration status not being reported correctly by vCenter.  See below for various checks to identify where this workflow may be failing.

Resolution

Validate vCenter Events and Automation Logging

vCenter must post the event so that the Aria Automation service account has permission to read them:

1. Check that the service account assigned to the Cloud Account can access vCenter and see the events.  Look for events in vCenter related to "VmRelocatedEvent" and "VmBeingRelocatedEvent".In Automation logs, check in the Provisioining-service-app logs for responses to these migrations that include "VmEmigratingEvent".

Checking that the event broker config maps contain the correct event types

The Event Broker service in Aria Automation must be properly patched and it's config maps must be updated.  If something failed during upgrade or the service has not been patched, we can see it by performing the following: 

1. SSH into Aria Automation.
2. Execute the following command:

   1. kubectl get configmaps -n prelude ebc-events-config -o json

Review the output for the "ebs.connect.enabled-vcenter-event-types" to ensure that we're collecting proper vCenter events. An example of this output:

"ebs.connect.enabled-vcenter-event-types": "VmMigratedEvent,VmEmigratingEvent,VmBeingRelocatedEvent,VmRelocatedEvent,VmRelocateFailedEvent,com.vmware.vc.vm.DstVmMigratedEvent,com.vmware.vc.vm.SrcVmMigratedEvent,com.vmware.vc.vm.SrcVmMigrateFailedEvent,VmFailedMigrateEvent"

 

1. Wait for data to sync and validate that the migration shows successful.

Validate the migration state is being populated in the Aria Automation database:

When migration is detected and a reconcile attempt is made, the resource metadata is updated with results under custom properties.  Check in the UI under the deployment/resource/custom properties for the properties called "migrationMessage" and "migrationStatus" and if there are errors they should show up in the migrationMessage.  Example of a successful reconcile will include the following status:

Alternatively you can query the metadata details of the resource in the database:

1. SSH into the Aria Automation appliance and execute the following commands:

   1. vracli dev psql provisioning-db

   2. \x

   3. select * from compute_state where name = 'COMPUTERNAME'

In the output, look for all instances of the custom properties related to 'migration' for information.