High procesisng Lag and Grid Usage is seen on Aria Operations for Networks 6.12.

Products

• VMware Aria Operations for Networks

Issue/Introduction

High processing Lag and Grid Usage is seen on Aria Operations for Networks version 6.12

Symptoms:

1. GUI Screenshots

   

   

 

2. Data bus is configured with 2 subscriber IDs one with HTTP and another with HTTPs. It is recommended to use HTTPS

 

3. Flink log exception is seen in log location: 

    

/var/log/hadoop-yarn/containers/application_1722602791013_0001/container_1722602791013_0001_01_000002 

 

Complete exceptions as below:

    

2024-07-30T05:17:26.153Z ERROR client.gateway.BackPressureFilter BackPressureFilter-scheduled processQueuedRequests:254 Error dispatching requests from the queue java.lang.NoSuchMethodError: 'void com.esotericsoftware.kryo.Kryo_DefaultInstantiatorStrategy.<init>(org.objenesis.strategy.InstantiatorStrategy)'
        at com.vmware.lemans.commons.base.serialization.KryoSerializers.create(KryoSerializers.java:61)
        at com.vmware.lemans.commons.base.serialization.KryoSerializers_KryoForDocumentThreadLocal.initialValue(KryoSerializers.java:249)
        at com.vmware.lemans.commons.base.serialization.KryoSerializers_KryoForDocumentThreadLocal.initialValue(KryoSerializers.java:244)
        at java.base/java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:195)
        at java.base/java.lang.ThreadLocal.get(ThreadLocal.java:172)
        at com.vmware.lemans.commons.base.serialization.KryoSerializers.getKryoThreadLocalForDocuments(KryoSerializers.java:145)
        at com.vmware.lemans.commons.base.serialization.KryoSerializers.clone(KryoSerializers.java:188)
        at com.vmware.lemans.commons.base.utils.Utils.clone(Utils.java:92)
        at com.vmware.lemans.client.gateway.GatewayOperation.setBody(GatewayOperation.java:53)
        at com.vmware.lemans.client.gateway.BackPressureFilter.buildGatewayRequest(BackPressureFilter.java:359)
        at com.vmware.lemans.client.gateway.BackPressureFilter.lambda_dispatchRequests_6(BackPressureFilter.java:288)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at com.vmware.lemans.client.gateway.BackPressureFilter.dispatchRequests(BackPressureFilter.java:287)
        at com.vmware.lemans.client.gateway.BackPressureFilter.processQueuedRequests(BackPressureFilter.java:227)
        at java.base/java.util.concurrent.Executors_RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor_ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor_Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829) 

 

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

Aria Operations for Networks 6.12.0
Aria Operations for Networks 6.12.1

Cause

Kafka's Topic3 partitions were not in sync and hence flink module was not able to read/write properly.

Resolution

This issue is fixed in Aria Operations for Networks version 6.13

For now we have a workaround to replace the Jar file.

Contact Broadcom Support by opening support case to obtain assistance with correctly replacing the Jar file

See Creating and managing Broadcom support cases.