How to Manually Complete a VMware Aria Operations for Logs Upgrade

Products

• VMware Aria Suite

Issue/Introduction

This article provides steps to complete the VMware Aria Operations for Logs upgrade manually in case it has failed/hung on one or more of the nodes.

Note: The Management > Cluster page shows one or more of the nodes in the cluster doesn't come back to a Connected state or its status is flapping between Connected and Disconnected.  This page won't reflect the correct status for the Primary node if it is in such a state.

Resolution

To continue the VMware Aria Operations for Logs upgrade via the command line, please complete the following steps.

1. Using an SCP utility like WinSCP, upload the upgrade PAK file to the /tmp folder of the problematic node.
2. Log into the problematic node as root via SSH or Console.
3. Run the following commands to manually start the Cassandra service:

• systemctl stop loginsight
• /usr/lib/loginsight/application/sbin/li-cassandra.sh --startnow --force

Note: Run the nodetool status command to verify that Cassandra is up and running.

4. Once the Cassandra service is up and running, run the following command to start the upgrade:

loginsight-pak-upgrade /tmp/pak_name
 

Note: Replace pak_name with the upgrade PAK file's name used in step 1.
Example: loginsight-pak-upgrade /tmp/VMware-vRealize-Log-Insight-8.12.0-21696970.pak

5. Run the following commands to reset the upgrade status in the Cassandra DB:

• version=8.12.0-21696970
• cqlsh-no-pass -e "update logdb.upgrade_status set status='Complete' where version='$version';"
• node_guids=`cqlsh-no-pass -e "select * from logdb.node_upgrade_status where version='$version';" | awk '{print $3}' | sort | uniq | head -n -2`
• for node in $node_guids; do query="update logdb.node_upgrade_status set status='Complete' where version='$version' and node_id='$node'"; cqlsh-no-pass -e "$query"; done

Note: The version value should match the version number from the PAK file used in step 1.

6. Repeat steps 1-5 on the remaining nodes in the cluster.

See the Related Information section for steps to re-apply the certificate to ensure the internode communication certificate gets updated.

Additional Information

After completing the manual upgrade steps in the Resolution section, ensure the cluster gets the updated internode communication certificate by re-applying the certificate.
Follow Install a custom certificate if you are using a custom certificate for VMware Aria Operations for Logs.
If you are using the default certificate, follow Revert to the default certificate.

Install a custom certificate

1. Log into the VMware Aria Operations for Logs as the local admin user.
2. Navigate to Configuration > SSL.
3. Click Choose File and select the certificate file you wish to use.

Note: VMware Aria Operations 8.12 and later requires Client Auth usage.  Ensure your custom certificate meets this requirement.

4. Click Save.

Revert to the default certificate

1. Log into the VMware Aria Operations for Logs UI as the local admin user.
2. Navigate to Configuration > SSL.
3. Click RESET TO DEFAULTS....

4. Click Reset.

Impact/Risks:
Starting with VMware Aria Operations for Logs 8.12, the upgrade process updates the internode communication certificate.  However, the certificate update only happens if the rolling upgrade is successful and won’t be triggered in case of a manual upgrade.  To ensure the cluster gets the updated internode communication certificate, see the Related Information section for instructions on re-applying the certificate after completing the Resolution steps in this article.