TCP connection/socket gets stuck and the handshaking is delayed

Support
1

Hi,

We have a client/server application which was developed a long time ago. It has been running in production for more than 10 years. The client is a Windows application written in C++, and the server-side component is written in Java8.

This client/server software has been working fine for a long time on Linux servers. Currently, we use AlmaLinux 9. It was working on AlmaLinux 9 until updating the kernel.

So, when we update the Linux kernel from “5.14.0-362.13.1.el9_3.x86_64” to “kernel-5.14.0-427.31.1.el9_4.x86_64” the application gets unstable: The client drops the connection based due to not receiving messages in the proper time. We notice delays, the client just waiting for the response from the server. The issue is always reproducible with the new kernel. And if we go back to the old kernel, the problem is gone. We kept running the test for hours in both cases.

I can provide PCAP files created by tcpdump tool in both cases: working and non-working scenarios.

Please investigate the issue that what happened between these two kernel versions. It seems there is an issue in the new kernel.

I already reported a bug on the kernel.org website.

Link: 219221 – TCP connection/socket gets stuck and the handshaking is delayed

You find the PCAP files there in the attachment.

Please analyze it, and try to figure out why the new kernel behaves differently and causes this weird behaviour.

Thanks a lot!

Regards,

Zoltan

2

Hi Zoltan, could you get a fix for this issue? I am suffering with similar problem and looking for a solution.

3

Hi, sorry for the late answer. Unfortunately, I did not get a solution so far. I hope newer kernels will solve it. I always test this scenario whenever we plan to upgrade our system with new kernels and OS packages.

4

I opened a Red Hat Jira ticket for this issue:

I included the affected kernel versions, the upstream bug report, and the TCP maintainer’s observation about the unusually small receive window.

Thanks.

5

The issue has been resolved.

The root cause was an extremely small server-side receive buffer configured by the legacy Apache MINA 2.0.0-M6 framework: SO_RCVBUF was only 1 KiB. Although the application configuration was identical, different kernel versions produced different effective TCP receive-window behaviour. On the affected newer kernel under VMware, the usable window sometimes fell to approximately 397 bytes, causing the Windows client to wait approximately five seconds before sending the remaining TLS data.

We explicitly increased the MINA socket receive buffer to 64 KiB before binding the listener:

{code}
acceptor.getSessionConfig().setReceiveBufferSize(64 * 1024);
{code}

After this change, the server negotiated Window Scale 1, the effective receive window remained at least approximately 64 KiB, and the 397-byte fragmentation, zero-window events and five-second delays disappeared.

Therefore, the newer kernel/VMware environment exposed a fragile application-level socket configuration, but the primary root cause was the excessively small MINA receive-buffer default. The exact reason why the two kernel versions derived different effective windows from the same 1 KiB buffer remains unidentified, but it no longer has a functional impact with the corrected buffer size.