2.5.9 -> 2.6.2 breaks my site

PMc

Hija,

after upgrading from 2.5.9 to 2.6.2 nothing works anymore.

Read here for details:
https://forums.freebsd.org/threads/upgrade-to-2023q2-openvpn-issue-www-freebsd-org-no-longer-reachable.88654/post-605306

From what I can say up to now, this is somehow related to IPv6 fragmentation.
AFAIK there is almost no usecase for IPv6 fragmentation, but I have a counter put into my OS that counts IPv6 fragments (because I need to temporarily reassemble them in order to feed them to the firewall). That counter was always at zero up to now, but with 2.6.2 installed it suddenly rises.

never-stop-learning

Hey man, what is the log shown to you? You can share it here.

PMc

Man, the log shows this:

suricata[39932]: [110405] <Warning> -- [ERRCODE: SC_WARN_IPFW_XMIT(84)] - Write to ipfw divert socket failed: Message too long
suricata[39932]: [109685] <Error> -- [ERRCODE: SC_ERR_FATAL(171)] - thread W-8677 failed

(And please leave it to my discretion what I can or cannot share on world-readable sites.)

After suricata has crashed, the site is obviousely offline. Nothing gets in, noting gets out. (I'll fix this in the suricata code as soon as I get around to it. The suricata guys stated that they don't want to support ipfw/divert integration any longer, so I'm on my own with that.)

Now, concerning the openVPN log, there is nothing of value about the message size calculation (with standard verb=3). Analysis is currently ongoing.

I am running this tunnel through IPv6 and HE tunnel (gif 6in4), so my header calculation is this: 8 UDP + 40 IPv6 + 20 IP/6in4 = 68.
The default header calculation in openVPN is 50, so the default settings cannot work here.

After ssh failed already at the KEX, I did full analysis in debug mode with version 2.5.6 (there are other nested tunnels tunneled thru this tunnel, so I need absolutely correct math), and figured out that I must set
link-mtu 1432
mssfix 1432
In that case, openvpn would configure the tun device as "mtu 1371", and tcp/IPv6 sessions would get mss=1311, and thing did work.
(The analysis -in german- is available on request, currently not public.)

Now with 2.6.2 (and unchanged config) these values have changed. The tun device is now "mtu 1380, and tcp/IPv6 sessions get mss=1322. That is blatantly wrong math, because:
1322 + 20 TCP + 40 IPv6 = 1382 (and not 1380). So in some cases the OS will fragment, in some cases there is Errno=40 as with suricata above.

The PMTU discovery, which is supposed to work with IPv6, doesn't work at all. Neither do websites honor it, nor does openVPN give a damn. I have read https://community.openvpn.net/openvpn/ticket/1452 and that doesn't look encouraging. Anyway, here it states "not supported" by OS (this is Berkeley unix).