TCP initial/default window size considerations and auto buffering on SG OS

Products

• ProxySG Software - SGOS

Issue/Introduction

1. Auto socket buffering was introduced in 7.3.x, and unlike the socket buffering mechanism in 6.7.5.x, allocates only as much memory as is needed to hold outstanding socket data at any given time, up to the max window-size you have specified
It means that the window-size is a max, but the socket buffers are limited until the applications using the sockets can prove that it can keep up. 

2. Without auto buff enabled, the tcp-stevens graph on wireshark is linear (what you see in 6.7), whereas the 7.3 has a slower start but exponentially grows for large file download via proxysg

a. On 6.7

b. On 7.3

3. Generally auto-buffer is a good thing; helps reduce buffer bloat; especially when the ProxySG has to handle hundreds of thousands of connections at once. 

4. Default window size considerations

SGOS 6.7.3.x and older - default TCP window size is 64K. 
Starting with SGOS 6.7.4.1 - default TCP window size has been increased from 64K to 256K. 
Starting with SGOS 7.2.1.1 - default TCP window size has been increased from 256K to 1MB

5. Initial window size considerations

a. TCP initial receiver window size (net.inet.tcp.recvspace) that proxysg uses depends upon the TCP window size (default 256K that is configured on proxysg, which can be checked by running command 'show tcp-ip' from proxysg CLI
b. In SG OS 7.3 (7.3.21 and below) and SG OS 7.4.5 and below, the TCP initial receiver window size (net.inet.tcp.recvspace) is 98K and this grows gradually. There is no option to disable auto socket buffering
c. In SG OS 7.3.21 and above and SG OS 7.4.6 and above, the TCP initial receiver window size (net.inet.tcp.recvspace) is 256K  and this grows gradually. Auto-socket buffering can be disabled 

6. Auto socket buffering considerations

a. In SG OS 6.7, auto socket buffering is disabled

b.  SG OS 7.3 (7.3.21 and below) and SG OS 7.4.5, auto socket buffering is enabled and there is no option to disable it

c. In SG OS 7.3.21 and above and SG OS 7.4.6 and above, auto socket buffering can be disabled (to revert to 6.7 behavior)

7. Note that due to above differences in the TCP stack in SG OS (6.7 vs 7.3), there may be differences observed in download speeds, especially for large file downloads

Resolution

Steps to disable auto socket buffering (SG OS 7.3.21 and above and SG OS 7.4.6 and above)

To enable or disable 'auto socket buffering' (it is enabled by default), issue the new configuration command:
# (config) tcp-ip tcp-auto-buffer {enable | disable}

Note:

a. This may increase the memory utilization on the EdgeSWG 

b. Also, ensure that the TCP window size is not set to more than 1MB (default), if auto socket buffering is disabled as this may add processing overhead.