Upgrading Aria Operations hangs on step 9 of 14 "Applied Operations System Update" and "Preparing to apply product update"

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

The Aria Operations upgrade has been stuck for 20 hours, with all nodes in the cluster showing "state: Not running" and "status: offline" in the admin UI.

Environment

8.12.1

Cause

• Either DNS was not configured, or it was corrupted or misconfigured.
• nslookup was not functioning during the upgrade.
• It may have been caused by the DNS servers not responding, which led to the upgrade being retried.

cd logs/casa-webapp/casa.log

Casa starts at 15:06

2025-02-27T15:06:21,221+0000  INFO [localhost-startStop-1] [        ] web.context.ContextLoader:257 - Root WebApplicationContext: initialization started

2025-02-27T15:06:21,753+0000  WARN [localhost-startStop-1] [        ] security.core.SpringSecurityCoreVersion:77 - **** You are advised to use Spring 5.3.29 or later with this version. You are running: 5.3.26

2025-02-27T15:06:22,399+0000  INFO [localhost-startStop-1] [        ] casa.support.CasaTokenAuthInterceptor:47 - Creating interceptor for the token auth inactiveSessionAliveMinutes= 60`

Shortly after, these messages start repeating frequently:

`2025-02-27T15:06:46,694+0000  INFO [taskScheduler-1] [        ] support.function.SystemFunction:67 - for ip=***************-***.********.********.**.** java.net.UnknownHostException: ***************-***.********.********.**.**: Temporary failure in name resolution`

Resolution

Steps:

• Check DNS Configuration: Ensure that the DNS settings are correctly configured on the vROps nodes. Verify that the /etc/resolv.conf file has the correct DNS server entries.
• Verify Network Connectivity: Ensure that the vROps nodes can access the DNS servers and there are no network issues preventing name resolution.
• Check /etc/hosts: If DNS is not available or intermittent, you might need to add the necessary hostnames and IP addresses to the /etc/hosts file on the vROps nodes.
• Verify that the vAAP properties are configured correctly and ensure that both forward and reverse DNS lookups are functioning properly.

Or

• If you have checked all DNS and gateway configurations in the environment and found no missing or mismatched settings, try resetting the DNS configuration on both the VM and vApp to rule out any intermittent issues. Then, reapply the configuration to the vROPs nodes.
• Then, attempt the upgrade again.