SSP Kafka Broker Closes Connections with "Buffer Overflow" Error, Causing NSX Manager Full-Sync Loops
Article ID: 450139
Updated On:
Products
Issue/Introduction
On SSP, the Kafka broker can proactively close a client connection mid-transmission - most visibly during NSX Manager full sync of config data. The Kafka broker log shows an IllegalStateException for an SSL buffer overflow, and NSX Manager (nsxapi.log) logs a corresponding Kafka disconnection and clears its message queue. If enough of these occur in sequence, NSX Manager's PaceAgent enters an error state and repeatedly re-triggers full sync, which may appear to loop and never complete.
Environment
- SSP 5.1.1 release
- Kafka broker/client connections using the FIPS BouncyCastle JSSE provider (ssl.provider=BCJSSE) with TLS 1.3 negotiated (ssl.enabled.protocols includes TLSv1.3) — the default SSP Kafka SSL configuration
- Most reliably triggered by large config payloads (e.g. NSX Manager PaceAgent full sync / config topics), since the defect only fires once a decrypted TLS record fills the buffer to capacity - small, compressed payloads (e.g. flow ingestion topics) are not affected
Cause
Log signature (for identification):
1. SSP Kafka broker — container/pod log for the affected kafka-controller-eligible-* / kafka-broker-* pod (console-only logging; capture via kubectl logs or the pod-log section of an SSP support bundle):
[2026-07-18 08:03:50,126] WARN [SocketServer listenerType=BROKER, nodeId=100] Unexpected error from / ##.##.#.# (channelId=##.##.#.#:9092-##.##.#.#:31000-6-1910); closing connection (org.apache.kafka.common.network.Selector)
java.lang.IllegalStateException: Buffer overflow when available data size (16384) >= application buffer size (16384)
at org.apache.kafka.common.network.SslTransportLayer.read(SslTransportLayer.java:617) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:85) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:462) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:412) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:679) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:581) ~[kafka-clients-4.1.1.jar:?]
at org.apache.kafka.common.network.Selector.poll(Selector.java:486) ~[kafka-clients-4.1.1.jar:?]
at kafka.network.Processor.poll(SocketServer.scala:1010) ~[kafka_2.13-4.1.1.jar:?]
at kafka.network.Processor.run(SocketServer.scala:914) ~[kafka_2.13-4.1.1.jar:?]
at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Both the internal (9092) and external/pace (9095) listeners can show this.
2. NSX Manager — /var/log/proton/nsxapi*.log on the manager node(s), timestamps line up with the broker-side exception above:
WARNING NSX ... [comp="nsx-manager" level="WARNING" logger="IntelligenceNsxCommunicationServiceImpl" msgID="INTELLIGENCE" subcomp="manager"threadName="message-sender-thread-####"] Kafka error occurred. Set error state and cleared message queue for cluster SSP.
INFO NSX ... [comp="nsx-manager" level="INFO" logger="IntelligenceNsxCommunicationServiceImpl" msgID="INTELLIGENCE" subcomp="manager"threadName="SSP-message-processor-#############"] Cluster is in active error state. Skipping message processing and clearing message queue.
The underlying client-side exception surfaced by the PaceAgent producer is:
org.apache.kafka.common.errors.NetworkException: Disconnected from node 100
3. Symptom on repeated occurrence — NSX Manager repeatedly re-triggers full sync without completing; FULL_SYNC_START / FULL_SYNC_END markers on thensx-config-* pods appear in a tight loop, and policynsxconfigepoch rows accumulate with fullsynccomplete=f, published=f for the affected site.
Cause:
Kafka's SslTransportLayer.read() sizes its decrypted-data buffer (appReadBuffer) to SSLEngine.getSession().getApplicationBufferSize() and nevergrows it beyond that. The FIPS BouncyCastle JSSE provider reports this size as exactly 16384 bytes (the max TLS record plaintext) with zeroheadroom; the reference SunJSSE provider reports 16709 (16384 + ~325 B of slack) that Kafka's read path implicitly relies on.
When a large, uncompressed message (e.g. NSX Manager's config full-sync payload) leaves undrained bytes in appReadBuffer and a subsequent full16384-byte TLS 1.3 record must be unwrapped, appReadBuffer.position() reaches the 16384-byte cap, unwrap() returns BUFFER_OVERFLOW, and - with no room to grow - Kafka throws IllegalStateException and closes the connection. Small, compressed payloads (e.g. flow-ingestion topics) rarely fill a record to exactly 16384 bytes and are effectively unaffected.
On the NSX Manager side, each broker-initiated disconnect puts PaceAgent into an error state and clears its queue; a linear-then-flat retry backoff(10 failures, then one retry per 60 minutes) means a burst of these disconnects during a large full sync can visibly stall/loop config sync for extended periods until the connection finally survives long enough to finish.
Resolution
Upgrade the SSP deployment to SSP 5.2.0 or later, which includes a fix (HeadroomSSLEngine wrapping the Kafka SSL engine in AnsSslEngineFactory) that makes the reported application buffer size include the headroom Kafka expects - restoring parity with SunJSSE - while keeping TLS 1.3 and the FIPS-mandated BCJSSE provider unchanged.
There is no supported permanent workaround on releases prior to 5.2.0, since the defect is in how the JSSE provider reports its buffer size, not in a tunable Kafka setting. If a customer is blocked and cannot upgrade immediately, reducing full-sync/group-sync producer parallelism (DeltaConfigProducerImpl thread count from 5 → 1 on NSX Manager) has been observed to reduce record fullness enough that full sync eventually completes, though the underlying buffer overflow can still occur - this is a mitigation, not a fix.
Recommend customers on SSP 5.x below 5.2.0 upgrade to SSP 5.2.0+.
Feedback
