BGP with MD5 encryption doesn’t work when shaping is turned on
search cancel

BGP with MD5 encryption doesn’t work when shaping is turned on


Article ID: 165403


Updated On:




While using BGP, many implementations of BGP include MD5 signature option for security. BGP uses TCP as a transport protocol. Every segment sent on a TCP connection for BGP uses the 16-byte MD5 digest produced by applying the MD5 algorithm to the TCP header.

PacketShaper with shaping enabled, changes the initial TCP window size in the three-way handshake,  which makes MD5 checksum to fail while the packet is received on the other side. This could also become a problem if you use a rate control policy on BGP.



Use the ip passthrough command to ignore the host(s) that prevent the changing of the window size and rate control from occurring if applied to the class or inherited.

If the host is on the inside, for example, then type:

ip passthrough inside src
ip passthrough outside dst

If the host is on the outside, for example, type:

ip pass outside src
ip pass inside dst

Use the ip show command to verify you are getting hits to the filter applied.

Next, create a file called startup.cmd and copy the same commands in this file. Make sure this file is located in the root directory 9.256/.

Note: BGP peers could be using a high MTU size; PS sitting in between may not be able to handle large frames. Please make sure the MTU on the BGP peers are set to 1500.