Health check on VMware Aria operations for Network reports alert “JournalNode service is not healthy on platform”

Products

• VCF Operations for Networks

Issue/Introduction

• The following alert is observed in the Aria operation for networks(AON) platform appliance GUI and all 3 platform VMs are part of a cluster.
  
• User updated the hostname via the standard Linux hostnamectl or by editing /etc/hostname directly.
• Following log symptoms are observed in journal logs (/var/log/arkin/jn_health.log) in the affected platform appliance.
  

  <Timestamp> ERROR hdfs-jn-sync-check.py MainThread hdfs-jn-sync-check.py:91 Problem:
  Traceback (most recent call last):
    File "/home/ubuntu/build-target/cdh/hdfs-jn-sync-check.py", line 88, in main
      journal_nodes_sync_check(get_cluster_connection_details(args.connectionconfig))
    File "/home/ubuntu/build-target/cdh/hdfs-jn-sync-check.py", line 55, in journal_nodes_sync_check
      timestamp_cmd_output = get_latest_timestamp(connection, nodes[nodename])
  KeyError: '<platform1>'

Environment

Aria Operations for Networks 6.14.x

Cause

Aria Operations for Networks, cluster configurations are distributed across multiple components. Changing a hostname using standard Linux commands (like hostnamectl or editing /etc/hostname) is an unsupported method because it fails to update the HDFS Metadata and Internal Inventory. 

Resolution

Follow this doc to update the hostname correctly. Login to platform-1 CLI using consoleuser account and run the following command.

hostname update --hosts platform1:<new_hostname1> platform2:<new_hostname2> platform3:<new_hostname3>