Automatic upgrade stops responding on some of the steps on the Update tab of VAMI

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• The automatic upgrade stops responding on some of commands/steps listed on the VAMI > Update tab.

Note: This screenshot is only for illustration purpose. 

• When examining the respective IaaS node's Management Agent All.log file, there are no errors.
• The last triggered commands from the time of the upgrade are all completed successfully.

Note: This issue is applicable only if the following conditions are true:

• There has been a VA failover event at any time before the upgrade,so that the former primary is now a replica
• Upgrade is from version older than 7.4 to version 7.5 (e.g. 7.3.1 --> 7.5)
• The replica upgrade has failed in the very beginning (e.g. at validation) so replica node is still the old version.
•  The IaaS automatic Upgrade is started, although the replica is not yet upgraded (this is allowed).

Environment

VMware vRealize Automation 7.4.x
VMware vRealize Automation 7.3.x
VMware vRealize Automation 7.5.x

Cause

This issue occurs because the MAs  continue to communicate with the Replica VA node which is still the old version. The upgrade script is triggered on the Primary VA node which is already the new version.

Resolution

Currently, there is no resolution.

Workaround:
To work around this issue, use one of these options:

Option 1:
Ensure the Replica node (or all replica nodes if more than one) are upgraded before proceeding with the automatic IaaS upgrade.
Note: If not explicitly excluded (by executing touch /tmp/disable-iaas-upgrade), the IaaS upgrade will start automatically after rebooting the primary VA (when the upgrade has completed).

Option 2:

1. Stop all Management Agent services on all IaaS nodes.
2. Open their respective ManagementAgent.exe.config files and ensure the list of VAs in the <managemetEndpoints> section starts with the current Primary VA.
3. Restart the Management Agents and trigger the automatic IaaS upgrade.

Option 3:
Upgrade IaaS manually using the IaaS installer that can be downloaded from the VAMI installer download page.