PathAnalyzer and PolicyManager Services are not running with the error: 'java.io.IOException: Failed to bind to address 0.0.0.0/0.0.0.0: <any port>" in the Aria Operations for Networks.

Products

• VCF Operations for Networks

Issue/Introduction

• The System Health section reports that one or more essential services are not running or either running but not healthy in the Aria Operations for Networks.
• You may see any services are not running or running but not healthy.
• Example in the below screenshot, the PathAnalyzer services is not running and PolicyManager running but not healthy.
  
  
  
  In the above example, the PathAnalyzer service is not running, so checking the path-analyzer logs located at /home/ubuntu/logs/path-analyzer on the affected platform show the following exception:
  
  

  YYYY-MM-DDHH:MM:SS ERROR veriflow.pathanalyzer.PathAnalyzerGRPCServer main main:149 Error booting up PathAnalyzer Server
  java.io.IOException: Failed to bind to address 0.0.0.0/0.0.0.0:47500
  at io.grpc.netty.shaded.io.grpc.netty.NettyServer.start(NettyServer.java:333) _[grpc-netty-shaded-1.63.0.jar:1.63.0]
  at io.grpc.internal.ServerImpl.start(ServerImpl.java:185) _[grpc-core-1.63.0.jar:1.63.0]
  at io.grpc.internal.ServerImpl.start(ServerImpl.java:94) _[grpc-core-1.63.0.jar:1.63.0]
  at com.vnera.veriflow.pathanalyzer.PathAnalyzerGRPCServer.start(PathAnalyzerGRPCServer.java:92) _[pathanalyzer-grpc-0.001-SNAPSHOT.jar:_]
  at com.vnera.veriflow.pathanalyzer.PathAnalyzerGRPCServer.start(PathAnalyzerGRPCServer.java:74) _[pathanalyzer-grpc-0.001-SNAPSHOT.jar:_]
  at com.vnera.veriflow.pathanalyzer.PathAnalyzerGRPCServer.main(PathAnalyzerGRPCServer.java:145) [pathanalyzer-grpc-0.001-SNAPSHOT.jar:_]
  Caused by: io.grpc.netty.shaded.io.netty.channel.unix.Errors_NativeIoException: bind(..) failed: Address already in use
  YYYY-MM-DDHH:MM:SS INFO veriflow.pathanalyzer.PathAnalyzerGRPCServer Thread-7 shutdown:98 Shutdown snapshotManager

Note: Identify the service that is not running from the System Health page, then check its respective logs for exception "Failed to bind to address 0.0.0.0/0.0.0.0".

Logs for each service are located under /home/ubuntu/logs within their respective service-named folders.

Environment

Aria Operations For Networks 6.14

Cause

In VCF Operations for Networks, each service binds to its designated port. However, these ports can sometimes be occupied by child jspawnhelper processes on the appliance, which prevents the services from running properly.

Resolution

A workaround is available, please open a support case with Broadcom. For more information, refer to Creating and managing Broadcom support cases.

This  issue is now fixed with Aria Operations for Networks 6.14 Patch6, Build number 1747413624

Aria Operations for Networks patch can be downloaded from Broadcom Support portal. 

To Download GA patch click here

File name: VMware-AriaOpNetworks.6.14.0.P6.1747413624.patch.bundle
File Size: 896 MB

Note: Above patch is cumulative of any previous patches for Aria Operations for Networks 6.14.0

Review below procedure and steps to apply Aria Operation for Networks Patch from GUI:

1. Download Aria Operations for Networks 6.14 Patch6 Build number 1747413624 on you local system.
2. Log into the Aria Operations for Networks GUI as an Administrator user. 
   
   Note:
   The default admin@local account can be used.
   Use platform node 1 IP in case of Clustered deployments.
   
   
3. Navigate to Settings > Infrastructure and Support> Infrastructure and Updates  then under Product, select Click here.
4. Click Browse to select the  locally downloaded patch file and click Upload.
   
   
5. In the Bundle Available message notification, click View details.
   
   Aria Operations for Networks (vRNI) Update screen appears.
   
   
6. Read the Before you proceed instruction and click Continue.
7. Wait for the pre-checks to complete.
8. Click Install Now.
   
   You can see the approximate time required to complete the update process on your setup.
   
   
9. Once the update process begins, the Aria operations for Networks Update screen provides the status of the upgrade  process.
10.  All platforms and the collectors nodes are updated.
    
    
    If you are using Aria Suite Life Cycle to Manage Aria Operations for Networks 6.14.0 then see below the Procedure to apply patch using VMware Aria Suite Lifecycle:
    
    Download Patch file here and click tab Solution Download to download the patch.
    
    File Name: vrlcm-vrni-6.14.0-6.14.0.P6.1747413624.patch
    File Size: 996.35 MB
     
    For applying the patch via VMware Aria Suite Lifecycle for VMware Aria Operation for Networks 6.14.0, refer to VMware Aria Suite Lifecycle 8.18 documentation for procedure and steps.