VPN connection dropping on a specific server.

nicolasg

Hi Guys,

I have installed OpenVPN 2.1.4 on many RedHat Linux, all seems to be working fine but some times the VPN connection goes down. This usually happens when I have to restart the OpenVPN server machine , some vpn clients just can connect and I have to manually run openvpn reopen or openvpn restart to re establish the connection.

Because of this issue I developed a script that automatically pings the VPN gateway and if the connection is down it will automatically re establish by running the above commands. Unfortunately in one of our servers the connection is going down very often, the script detects it and re-stablih the connection but after some seconds connectivity goes down again. This happens in a completely random time and not for a stable period, for example it can happen one day for 12 hours trying to re establish a connection or it can happen after a week for 1 hour..

Checking the logs at the server side the only thing I see suspicious is the message "Fri Apr 15 08:57:06 2011 us=169632 engine1/x.x.x.x:40698 MULTI: bad source address from client [10.11.0.10], packet dropped"

We have many servers running with the exact same OpenvVPN configuration and with the same hardware details but the problem seems to occur only on a specific one.

Any ideas on what can be wrong here? My manager want to switch to IPSec and I'm doing my best to keep OpenVPN as our permanent solution...

janjust

the message

"Fri Apr 15 08:57:06 2011 us=169632 engine1/x.x.x.x:40698 MULTI: bad source address from client [10.11.0.10], packet dropped"

is caused when a client behind the machine 'engine1' is trying to send traffic via the tunnel. It should not be the root cause of why this VPN connection is dropping.

The real question is why 'engine1' is kicked out the VPN after such a short time - what do the logs show on client and server side? In a LAN setup I would suspect a bad ethernet cable, in a WAN setup I would suspect a bad ISP connection.

nicolasg

Hi janjust,

Thank you for your reply. I'm afraid none of those is the case as all engines we have are behind the same load balancer and the monitoring system doesn't indicate any connectivity problem.

What exactly do you mean that a client behind engine1 is trying to send traffic via the tunnel ? if this was the problem then I can understand this client having problems to reach his destination address but we are talking here that the VPN connection goes down and when I manually try to recovery by restarting the service it only re-establish the connection for a very short time (less than a minute).

On the client side logs I can only see my script running and the VPN negotiation successful, on the server side the only suspicious is the "packet dropped" ..

Any other ideas ?

janjust

which machine has the "rogue" address 10.11.0.10?

if the reconnect happens too quickly the server might not know that the client connection was dropped. It should see the reconnect attempt, however.

The root cause remains why the client drops out so often.
Which message is shown when client drops the connection (again).

nicolasg

The 10.11.0.10 is the engine1 VPN address.

On both server and the clients the logs doen't show anything apart from the usual VPN conneciton sequence when the service is restarted. The only thing that differs is the :

Fri Apr 15 08:57:06 2011 us=169632 engine1/x.x.x.x:40698 MULTI: bad source address from client [10.11.0.10], packet dropped

janjust

are you using

keepalive 10 60

or similar on the openvpn server at all? this often helps to reduce flapping lines...

nicolasg

I'm using

keepalive 10 120

janjust

ah OK, so we can rule that out ; are you using 'proto udp' or 'proto tcp' ? As I said before, the root cause of your problem is the fact that this one client's connection is flapping up and down - esp since the server does not seem to be aware of this flapping.

nicolasg

I'm using proto udp.

As I said before, the root cause of your problem is the fact that this one client's connection is flapping up and down - esp since the server does not seem to be aware of this flapping.

VPN connectivity is for sure flapping up and down but network connectivity is not as we didn't had such indication in our network monitoring system.
We have many machines in the VPN and from time to time we see some clients disconnecting but reconnecting successful because our VPN script take care of it. It's just for that specific machine even when we manually restart the service the VPN connection goes down very fast resulting in a long downtime..

Thanks for your time with this.

janjust

but what message does the client display (with --verb 5) when the connection goes down so quickly?

nicolasg

hmm didn't thought about that , the current verbosity is on 3. I'll change it to 5 and update the post when the problem occurs again.