Route between clients drops regularly.

rew1

I have a setup where i use openvpn for the classical use of "VPN": virtual private network. I have my own server on the internet.

I have two locations with a bunch of computers at each location. These use /24 networks.
I have a few random computers "in the field" that also need to be on the VPN. Among these is a server that handles my outgoing Email. These use ip addresses from a third /24 network.

Now someone every now and then the route between where I send the EMail (on one of the /24 nets) and the server (on a single host VPN connection to the server) goes away. The initiating a ping on the server to the host on my private net will trigger that it suddenly starts working again. abra2:~> ping 192.168.6.14 PING 192.168.6.14 (192.168.6.14) 56(84) bytes of data. 64 bytes from 192.168.6.14: icmp_seq=10 ttl=63 time=5.88 ms 64 bytes from 192.168.6.14: icmp_seq=11 ttl=63 time=5.65 ms 64 bytes from 192.168.6.14: icmp_seq=12 ttl=63 time=5.84 ms
So here I was: "lets ping the server and see why it isn't working?" So it wasn't working. Once this ping was running I logged into the server using its public IP address and started a "ping abra2"... Appartently that cost me between 9 and 10 seconds and then it started to work again.

So why is the openvpn server forgetting the "route" to the server?

It seems that normal openvpn routing between the clients does not go through the system routing tables, but stays internal to openvpn. This makes it difficult for me to debug this. I can't tcpdump the packets to see what's going on.
As I need the connection for my outgoing mail, and mail is "used to" links being down, it can be a long time before I notice my mails no longer being sent to the sever....

Keeping a ping -i 60 running on the internet server seems to work until I accidentally close that window.

The Johan

OpenVPN does have an internal routing table for client-to-client traffic. You can still do packet capturing between the OpenVPN clients as they need to put packets into the tunnel and getting them out again too.

When I see in networking that routes stop working and then suddenly start working when you ping them, that's either a firewall deciding to be annoying (allow it from your computer and allow traffic back for a limited time) or you have some conflicting routes on the computers so the OS ends up picking the wrong one occassionally.

rew1

As to the "in and out"... I can see packets going towards the server, the server then doesn't route them, and then I don't see them coming out on the other VPN link. How can I check where things are going wrong on the openVPN server that's supposed to be routing those packets?

I don't think it is a firewall rule. Of course I don't care about ping packets. What I care about is that my mail gets forwarded. When it doesn't get forwarded I start debugging with ping and other tools.

The openVPN server currently doensn't have ANY firewall rules. Maybe not the best idea in the world. I should fix that one day, but for now iptables -L comes up empty and we can rule that out.

Oh, with "firewall" you mean a consumer grade router between "local network" and the ISP's network? Those often come preconfigured to drop "established" connections when they've been idle for more than say 300 seconds. Right!

The connection that is causing problems has "a server on the internet" with VPN to "another server on the internet (the VPN server)".
The other end of the problem is a server on a LAN with a small Linux computer as the router towards the ISP. I've removed the ISP's "consumer grade shit". The small linux computer has the VPN to the VPN server.
To be sure: when the route drops, everything can be verified working, So VPN from the small linux router to another lan via the VPN server: works fine! It is just that from the LAN to the other single server... it is not working.

I really think it is the openvpn server that is forgetting the route. Can I debug this by making the openvpn server log its routing decisions, or maybe when it gets packets that can't be routed?