Multiple error events at vSphere hosts "The host "<FQDN of Aria Operations for logs Node Name>:514" has become unreachable. Remote logging to this host has stopped"

Products

• VMware Aria Suite

Issue/Introduction

• ESXi hosts will observe following events in vCenter Server "The host "<FQDN of Aria Operations for logs Node Name>:514" has become unreachable. Remote logging to this host has stopped"
  
  
• Thread exhaust limit was identified from log-forwarder.log within the Cloud Proxy VA
  
  2024-09-11 07:12:51 [vertx-blocked-thread-checker] WARN  i.v.c.i.BlockedThreadChecker:? - Thread Thread[vert.x-eventloop-threa
d-0,5,main] has been blocked for 676410 ms, time limit is 2000 ms
io.vertx.core.VertxException: Thread blocked
        at [email protected]/java.net.SocketOutputStream.socketWrite0(Native Method)
        at [email protected]/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
        at [email protected]/java.net.SocketOutputStream.write(SocketOutputStream.java:129)
        at [email protected]/java.io.DataOutputStream.writeBytes(DataOutputStream.java:276)
        at app//com.vmware.log.forwarder.processors.PostToTcpProcessor.postPayloadToTcpEndpoint(PostToTcpProcessor.java:215)
        at app//com.vmware.log.forwarder.processors.PostToTcpProcessor.process(PostToTcpProcessor.java:97)
        at app//com.vmware.log.forwarder.services.EventForwardingService.postToTcp(EventForwardingService.java:73)
        at app//com.vmware.log.forwarder.services.EventForwardingService.handleForwardEventsRequestBody(EventForwardingService
.java:124)

  
  
  
• Service proxy running out of memory 
  
  
  

Environment

Aria Operations for Logs ( SaaS )

Cause

The cause of the "Thread exhaustion limit" in the log-forwarder.log within the Cloud Proxy VA may be due to an overwhelming number of simultaneous requests or connections that exceed the configured thread pool capacity. This can happen during peak load times or due to inefficient log processing.

Resolution

To resolve this condition:

• Increase the resource as per the Cloud Proxy Resource Limits 
• Increase the Resources for a Service Proxy as per the Modify Resources for a Service Proxy document