search cancel

Upgrade API Gateway 9.3.00-CR03 to 9.4.00-CR01 - error - Problem while reading DataSerializable

book

Article ID: 134821

calendar_today

Updated On:

Products

API SECURITY CA API Gateway Precision API Monitoring Module for API Gateway (Layer 7) CA API Gateway Enterprise Service Manager (Layer 7) STARTER PACK-7 CA Microgateway CA API Gateway

Issue/Introduction

Upgrade fails on CA API Gateway from version in our case 9.3 to version 9.4 with error:

 

2019-11-13T16:26:29.810+0100 SEVERE  106 com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl: [x.x.x.x]:8777 [gateway] [3.10.2] Failed to process packet: Packet{partitionId=-1, conn=Connection[id=1, /x.x.x.x:14490->/x.x.x.x:8777, endpoint=[x.x.x.x]:8777, alive=true, type=MEMBER], rawFlags=10001, isUrgent=true, packetType=OPERATION, typeSpecificFlags=[isResponse=false, isOpControl=false]} on hz.gateway-hazelcast.priority-generic-operation.thread-0
com.hazelcast.nio.serialization.HazelcastSerializationException: Problem while reading DataSerializable, namespace: 0, ID: 0, class: 'com.hazelcast.cluster.impl.operations.JoinCheckOperation', exception: com.hazelcast.cluster.impl.operations.JoinCheckOperation
 at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.rethrowReadException(DataSerializableSerializer.java:180)
 at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:161)
 at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:105)
 at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.read(DataSerializableSerializer.java:50)
 at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
 at com.hazelcast.internal.serialization.impl.AbstractSerializationService.toObject(AbstractSerializationService.java:187)
 at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:322)
 at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:390)
 at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:115)
 at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:100)
Caused by: java.lang.ClassNotFoundException: com.hazelcast.cluster.impl.operations.JoinCheckOperation
 at com.l7tech.server.policy.module.j.findClass(Unknown Source)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at com.l7tech.server.policy.module.j.loadClass(Unknown Source)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:173)
 at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:147)
 at com.hazelcast.nio.ClassLoaderUtil.newInstance(ClassLoaderUtil.java:101)
 at com.hazelcast.internal.serialization.impl.DataSerializableSerializer.readInternal(DataSerializableSerializer.java:150)
 ... 8 more
Caused by: java.security.PrivilegedActionException: java.lang.ClassNotFoundException: com.hazelcast.cluster.impl.operations.JoinCheckOperation
 at java.security.AccessController.doPrivileged(Native Method)
 ... 16 more
Caused by: java.lang.ClassNotFoundException: com.hazelcast.cluster.impl.operations.JoinCheckOperation
 at com.l7tech.server.policy.module.c.run(Unknown Source)
 at com.l7tech.server.policy.module.c.run(Unknown Source)
 ... 17 more
Caused by: java.lang.ClassNotFoundException: com.hazelcast.cluster.impl.operations.JoinCheckOperation
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at com.l7tech.server.policy.module.j.access$301(Unknown Source)

 

 

 

 

 

Environment

Release :

Component : API GATEWAY

Cause

The root cause of this problem during the upgrade is caused by the difference,  in the hazelcast library between the cluster nodes.

The newer version can not speak to the other node due to this.

Resolution


The workaround is  to make sure all nodes are down during the upgrade.

or    comment out the following line in system-properties until all nodes are upgraded 


like this : 


# Default to HazelcastEmbeddedSharedStateProvider


com.l7tech.server.extension.sharedKeyValueStoreProvider=embeddedhazelcast