[Solved] Suddenly getting Message too long

erikvl

I have an installation running many clients, about 240 connected right now. It has been running for many years with a growing number of clients. Recently I noticed "Message too long (code=90)" messages and investigated.

I am aware of the suggestions about --fragment and --mssfix. They have never been needed before.

Last Friday, these messages disappeared and today they suddenly reappeared. Nothing about the configuration has been changed.
I can't seem to find a correlation with a specific client and the syslog lists a number of candidates (write UDPv4), all apparently random and certainly not recently (re)connected.

Most of the messages look like this:

read UDPv4 [EMSGSIZE Path-MTU=1452]: Message too long (code=90)

or

read UDPv4 [EMSGSIZE Path-MTU=1452|EMSGSIZE Path-MTU=1452]: Message too long (code=90)

or

cc160-127/94.211.138.227:49148 write UDPv4 [EMSGSIZE Path-MTU=1452|EMSGSIZE Path-MTU=1452]: Message too long (code=90)

The write messages contain the clients common name and IP. I have tried a few and none seem to be having communication problems (I can SSH just fine and ping with different sizes succeed until the expected maximum [MTU is set to 1500] is reached).

Clearly something has changed, but I can't find what. Not the server or clients, that much is certain. Neither has the network where the server is located.

I am especially baffled by why it disappeared last Friday and reappeared just now.

I will be contacting the provider at the server end, maybe something is going on there. There is absolutely no control over the clients.

Anyone any suggestions?

erikvl

I solved this issue by doing a packet trace on the OpenVPN server.

I was able to trace ICMP type 3 code 4 messages back to a single IP and at that IP I was able to locate a client the had connected just before when the messages started. Turns out this was the problematic client. I just blocked it to see what happens next.

It does make me wonder. If the destination returns a proper Frag needed response, why does OpenVPN not adjust properly? I am using version 2.2.1.

Maybe every other client supports an MTU of 1500 and this is the only one that doesn't? Or is this a special case?

Maybe the actual client, which is masqueraded behind this public IP, has an even smaller MTU size and does not respond properly?

TinCanTech

erikvl wrote I am using version 2.2.1.

Dated approximately 08-May-2013 ........................ :roll:

erikvl

TinCanTech wrote
erikvl wrote I am using version 2.2.1.
Dated approximately 08-May-2013 ........................ :roll:

I know. Silly, right.

This is a platform with hundreds of users running on Ubuntu 12.04 LTS. Not something that can be replaced easily, although an addition in the form of a second server running an additional OpenVPN server will be released within a few months. Obviously that will be running Ubuntu 16.04 LTS with an updated OpenVPN and since it will be a load-balancing solution, it will allow for a systems update of the current server without the risk of downtime.

So maybe this is all OpenVPN's fault and it will all be solved after the update.

TinCanTech

erikvl wrote maybe this is all OpenVPN's fault and it will all be solved after the update.

TinCanTech wrote
erikvl wrote I am using version 2.2.1.
Dated approximately 08-May-2013 ........................ :roll:

You have had ample time to update your server .. 4.5 years in fact.

erikvl

TinCanTech wrote You have had ample time to update your server .. 4.5 years in fact.

You know all the circumstances involved how?

Besides, 14.04 is not 4,5 years ago. You don't just upgrade immediately either.

TinCanTech

I am just going by the details you posted .. Openvpn 2.2.1 is 4.5 years old .. I am not discussing your ubuntu OS.

Perhaps this will be of use in future:
https://community.openvpn.net/openvpn/wiki/OpenvpnSoftwareRepos

erikvl

TinCanTech wrote I am just going by the details you posted .. Openvpn 2.2.1 is 4.5 years old .. I am not discussing your ubuntu OS.

I see. I am fully aware that the server is running a very old version by now, but as you may know that when something isn't broken, then not fixing it is always a good rule of thumb.
So far nothing had been broken (and still might not be) an there was no need for any of the new features of OpenVPN.
Even the new feature of load-balancing currently in development for the platform does not require any of the new features. A testament to OpenVPN's features no doubt.

TinCanTech wrote Perhaps this will be of use in future:
https://community.openvpn.net/openvpn/wiki/OpenvpnSoftwareRepos

It will, thanks. Maybe I will update OpenVPN separately of the entire OS. That will certainly reduce the risk to next to nothing.

TinCanTech

erikvl wrote when something isn't broken, then not fixing it is always a good rule of thumb

Openvpn 2.2.1 will almost certainly be broken unless the ubuntu maintainers have back ported a lot of security fixes.

Post the output of:

$ openvpn --version

erikvl

Its:

OpenVPN 2.2.1 x86_64-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on Dec 1 2014
Originally developed by James Yonan
Copyright (C) 2002-2010 OpenVPN Technologies, Inc. <sales@openvpn.net>

$ ./configure --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --libexecdir=${prefix}/lib/openvpn --disable-maintainer-mode --disable-dependency-tracking CFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security CPPFLAGS=-D_FORTIFY_SOURCE=2 CXXFLAGS=-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security FFLAGS=-g -O2 LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now --enable-password-save --host=x86_64-linux-gnu --build=x86_64-linux-gnu --prefix=/usr --mandir=${prefix}/share/man --with-ifconfig-path=/sbin/ifconfig --with-route-path=/sbin/route

Compile time defines: ENABLE_CLIENT_SERVER ENABLE_DEBUG ENABLE_EUREPHIA ENABLE_FRAGMENT ENABLE_HTTP_PROXY ENABLE_MANAGEMENT ENABLE_MULTIHOME ENABLE_PASSWORD_SAVE ENABLE_PORT_SHARE ENABLE_SOCKS USE_CRYPTO USE_LIBDL USE_LZO USE_PF_INET6 USE_PKCS11 USE_SSL

TinCanTech

erikvl wrote OpenVPN 2.2.1 x86_64-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on Dec 1 2014

Take note:
https://community.openvpn.net/openvpn/wiki/SecurityAnnouncements