The log message: “Unable to instantiate or initialize instance hdfs-##-####” is safely ignorable on single-instance AON Platform VM

Products

• VCF Operations for Networks

Issue/Introduction

• The platform VM is deployed as a single node.
• On a single-instance platform VM, the following log messages are frequently recorded in /var/log/syslog. 
  

  MMM DD HH:MM:SS ##### agent[####]: YYYY-MM-DD HH:MM:SS UTC | JMX | WARN | App | Unable to instantiate or initialize instance hdfs-dn-#####. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
  MMM DD HH:MM:SS ##### agent[####]: YYYY-MM-DD HH:MM:SS UTC | JMX | WARN | App | Unable to instantiate or initialize instance hdfs-nn-#####. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
  MMM DD HH:MM:SS ##### agent[####]: YYYY-MM-DD HH:MM:SS UTC | JMX | WARN | App | Unable to instantiate or initialize instance hdfs-jn-####. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:
  MMM DD HH:MM:SS ##### agent[####]: YYYY-MM-DD HH:MM:SS UTC | JMX | WARN | App | Unable to instantiate or initialize instance hdfs-zkfc-####. Is the target JMX Server or JVM running? Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is:

• Log in to the target platform VM as consoleuser and execute the “show-service-status” command. It will display “All required services are running”.
  

  (cli) show-service-status
  ElasticSearch is running
  HMaster is running
  HRegionServer is running
  QuorumPeerMain is running
  ResourceManager is running
  NodeManager is running
  Launcher is running
  VIPService is running
  DatabusGateway is running
  FlinkContainer is running
  SaasListener is running
  Kafka is running
  Restapilayer is running
  TSDB is running
  Nginx is running
  ExpressJSApp is running
  NTPSEC is running and healthy.
  FoundationDB is running
  All required services are running. <-----! 
  (cli)
  

 

Environment

VMware Aria Operations for Networks

Cause

• The HDFS-related services, which these logs pertain to, are only fully configured in clustered deployments, not on a simple, single-node platform deployment. 
  
  
• The logging is related to a monitoring agent health check attempting to connect to HDFS components on a single-node setup where those components or their JMX interface may not be available.
  
  

Resolution

The log messages can be safely ignored as the system remains healthy.
HDFS does not exist on simple single-node deployments, and all required services appear to be running correctly.