OpenVPN within OpenVPN, MTU issues

[akorn]

Hi,

I'm transporting an OpenVPN UDP tunnel within another OpenVPN UDP tunnel, and am experiencing MTU issues I don't understand.

The inner tunnel uses TAP devices and has an MTU of 1500. It's part of an Ethernet bridge, so ideally I'd like to keep its MTU high (1500 may even be too low since there can be Ethernet frames that won't fit).

The outer tunnel uses TUN devices and also has an MTU of 1500.

The topology looks like this:

Code: Select all

+----------+     +----------+                          +----------+     +----------+
| intranet +-----+ router A +---<openvpn in openvpn>---+ router B +-----+ Internet |
+----------+     +----------+                          +----------+     +----------+

Router A and router B can ping each other using 1500-byte packets over the innermost OpenVPN link (ping -M do -s1472 other-router works). This could be a result of comp-lzo being able to compress the ping packets to be smaller.

When connecting to an https server on the intranet from the Internet (with the connection being forwarded via OpenVPN), the SSL handshake hangs. I can see packets like this in tcpdump on router A, entering the inner tunnel:

Code: Select all

09:18:15.637834 IP (tos 0x0, ttl 63, id 13034, offset 0, flags [DF], proto TCP (6), length 1296) 10.74.91.26.443 > 162.21.20.57.47880: Flags [.], cksum 0x129a (correct), seq 1:1245, ack 518, win 59, options [nop,nop,TS val 1110009216 ecr 948988316], length 1244

According to tcpdump on router B, they don't come out of the inner tunnel. I wonder who is dropping them?

If I lower the MTU on the inner OpenVPN link to 1292 on router A, everything works. Larger values don't work.

The outer tunnel is established over physical links that have an MTU of 1500. Both routers run Linux.

(In case you're wondering, the double tunnel is needed because the outer tunnel uses route-based failover and load-balancing, which must be transparent to the bridge the inner tunnel is part of.)

There are a number of things I don't know:

1. what happens to frames larger than 1292 bytes router A receives on its physical Ethernet, that's part of the same bridge as the inner OpenVPN TAP device? If I lower the MTU on the TAP device to 1292, the MTU of the entire bridge changes to 1292. Can it still receive larger frames, just not send them, causing any that would need to be bridged to be dropped?

2. why are packets being dropped silently?

3. how should I set mtu, fragment, mssfix, tun-mtu etc. to be able to forward 1518-byte Ethernet frames in the inner UDP tunnel?

Thanks!

[akorn]

Interesting data point: the inner tunnel used to use TLS; I switched it to a shared-secret based one and can no longer reproduce the MTU issue. Colour me puzzled.

[akorn]

I spoke too soon. I can no longer reproduce the MTU issue with the SSL handshake, but this packet, for example, still didn't get through:

Code: Select all

14:16:01.125070 00:16:3e:c3:19:da > 26:73:d8:c8:aa:48, ethertype IPv4 (0x0800), length 1314: (tos 0x0, ttl 63, id 48219, offset 0, flags [DF], proto TCP (6), length 1300)
    10.74.91.26.80 > 162.21.20.57.57028: Flags [P.], cksum 0xc6fb (correct), seq 160655:161903, ack 10157, win 145, options [nop,nop,TS val 1114475588 ecr 953480655], length 1248

Setting the mtu to 1300 (so, not 1292) seems to work now, but I'd still prefer an MTU of 1518 for the inner tunnel.

[akorn]

Setting the internal tunnel to `tun-mtu 1518`, `fragment 1300` seems to do the trick.

Now I'd just like to know how to arrive at `fragment 1300` analytically instead of trial and error.

[Pippin]

This might interest you:
"mssfix max is the guide broken"
https://sourceforge.net/p/openvpn/mailm ... vpn-users/