This article describes the Non-sticky Connections check box in the 3rd Party Configuration section of the cluster object properties in the Check Point Management GUI and when it should be used.Large delay when comparing time stamps for ingress traffic and egress traffic after Firewall inspection.
In addition, large amount of traffic retransmission could be seee on the network.

Problem:

When "Support for non-sticky connections" feature is enabled under the cluster object --> 3rd party configuration,  checkpoint enables a mechanism that is called "Flush & Ack".



Non-sticky  (or non-persistent)  connections are connections in which the return packet is handled by a different cluster member than the original packet. Sticky connections are all connections where packets in both directions are handled by a single cluster member.

Support non-sticky connections – also known as “flush’n’ack”, is a setting that Check Point can use to handle "non-sticky" connections. In effect, this setting forces a cluster member to hold the first packet of any new connection until it either receives an acknowledgement from every other cluster member that the connection is synced or the timeout period for holding the packet expires, in which case it is added to the connections table.

The idea behind this setting is that if connection stickiness through the cluster cannot be guaranteed, then if a node in the Check Point cluster receives an entry that may be “out of state”, it will hold onto the connection until state sync has completed. A packet may be “out of state” because the original packet went through another cluster member, and the subsequent packets are received before the state sync has completed.

The "Support Non-Sticky Connections" option adds significant firewall overhead in particular configurations. For example, when a cluster member (even on a backup chassis) is slow to respond to sync requests or is down, all synced connections will be held until the timeout period expires, resulting in high latency on all cluster members. This symptom is often observed by seeing the first packet in a connection take a very long time (such as first ping is lost, first DNS request is slow, etc). 

Since the Crossbeam NPM guarantees flow stickiness, the option "Support Non-Sticky Connections" in the 3rd Party Configuration of the cluster object should be unchecked except in certain uncommon situations as recommended by Support or documentation. There are two primary reasons for this:

1. Checking this option will place additional load on the cluster members which isn't necessary since the Crossbeam X-Series Platform handles connection persistence independent of the cluster.

2. In the event that a cluster member (even on a backup chassis) is slow to respond to sync requests or is down, all synced connections will be held until the timeout period expires, resulting in high latency on all cluster members.

Workaround