OpenVPN connection does not always work for some clients

[NewtownGuy]

I run three OpenVPN servers on two ISP's. Two of them, which are on the same Internet connection, always accept connections from all clients. But the third one, which is on a different ISP, sometimes won't let certain clients connect all the time, even though its Internet connection is by far the fastest (10+ Mbps). It is not reliable -- sometimes a problematic client can connect but sometimes that same client cannot connect. I've been trying to find the cause for months but have not been able to.

I have set the logging level on both client and server to 6 as suggested.

I found that using the 'float' option in the client's config file helped, even though the public IP address of the server never changes even though it is technically dynamic.

Here's part of the sanitized log file from the problematic client, where you can see many RESETS at the end:

Sun Jan 16 07:28:30 2011 us=248535 remote_float = ENABLED
Sun Jan 16 07:28:30 2011 us=248569 ipchange = '[UNDEF]'
Sun Jan 16 07:28:30 2011 us=248602 bind_defined = DISABLED
Sun Jan 16 07:28:30 2011 us=248634 NOTE: --mute triggered...
Sun Jan 16 07:28:30 2011 us=248677 185 variation(s) on previous 20 message(s) suppressed by --mute
Sun Jan 16 07:28:30 2011 us=248716 OpenVPN 2.1_rc7 i486-pc-linux-gnu [SSL] [LZO2] [EPOLL] built on Dec 15 2010
Sun Jan 16 07:28:30 2011 us=248826 WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
Sun Jan 16 07:28:30 2011 us=249727 WARNING: file '/etc/openvpn/keys/client.key' is group or others accessible
Sun Jan 16 07:28:30 2011 us=250876 /usr/bin/openssl-vulnkey -q -b 1024 -m <modulus omitted>
Sun Jan 16 07:28:30 2011 us=516230 LZO compression initialized
Sun Jan 16 07:28:30 2011 us=516498 Control Channel MTU parms [ L:1542 D:138 EF:38 EB:0 ET:0 EL:0 ]
Sun Jan 16 07:28:30 2011 us=516645 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:135 ET:0 EL:0 AF:3/1 ]
Sun Jan 16 07:28:30 2011 us=516723 Local Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'
Sun Jan 16 07:28:30 2011 us=516757 Expected Remote Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
Sun Jan 16 07:28:30 2011 us=516830 Local Options hash (VER=V4): '41690919'
Sun Jan 16 07:28:30 2011 us=516883 Expected Remote Options hash (VER=V4): '530fdded'
Sun Jan 16 07:28:30 2011 us=517486 Socket Buffers: R=[110592->131072] S=[110592->131072]
Sun Jan 16 07:28:30 2011 us=517630 UDPv4 link local: [undef]
Sun Jan 16 07:28:30 2011 us=517714 UDPv4 link remote: 24.a.b.c:1194
Sun Jan 16 07:28:30 2011 us=518023 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:32 2011 us=618507 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:34 2011 us=716026 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:36 2011 us=816061 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:38 2011 us=915985 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:41 2011 us=16156 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:43 2011 us=216147 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0
Sun Jan 16 07:28:45 2011 us=415991 UDPv4 WRITE [14] to 24.a.b.c:1194: P_CONTROL_HARD_RESET_CLIENT_V2 kid=0 [ ] pid=0 DATA len=0

..................

Here's a sanitized version of client.conf:

client

dev tun

proto udp

float

remote 24.a.b.c 1194

resolv-retry infinite

nobind

persist-key
persist-tun

ca /etc/openvpn/keys/ca.crt
cert /etc/openvpn/keys/client.crt
key /etc/openvpn/keys/client.key

comp-lzo

logfile foo.log

verb 6

mute 20

....................

I considered trying TCP, but the posts said it's a bad idea.

What should I do ?

Thank yoiu.

-- NewtownGal

[maikcat]

yes using tcp its not the ideal ,but is works...

why using float on your client?

cheers,

michael

[NewtownGuy]

I'm using float on the client because it seems to make more reliable connections from machines that have a hard time connecting. I do not see why this should be, but it seems to help sometimes.

However, I currently have a distant machine that would connect sometimes, but it just won't connect at the moment to an OpenVPN server on one of my two ISP's. That same machine connects just fine using UDP to two OpenVPN servers on a different ISP. So it looks like there is a problem with one of the local ISP's, although other machines can connect to it ok.

That makes me think there is a timing problem in OpenVPN when it uses UDP.

I have numerous machines connecting to an OpenVPN server via UDP. If I set up some OpenVPN clients to use TCP, don't I have to create an additional instance of OpenVPN server to use TCP instead of UDP ? If so, can I have both instances share the same /etc/openvpn folders (Ubuntu) and how do I start the additional instance ?

--->>> What measurement can I run to help diagnose why some machines have such a hard time connecting via UDP ?

-- NewtownGuy

[gladiatr72]

Hello,

What does it look like when you run a traceroute from an affected client and the problematic server? This sounds like a situation where, despite bandwidth being provided to the server by your ISP, their connectivity to the Internet is somewhat flaky.

As far as setting up a TCP server instance under ubuntu, you can do so by simply copying your existing server.conf file to (for instance) server_tcp.conf and adding "proto tcp-server" and "proto tcp-client" to the client.conf file. You can then add the name, minus the .conf extension, to the AUTO_START variable in /etc/default/openvpn to spin up that server instance on boot.

The following article describes some of the issues with TCP-over-TCP configurations: http://sites.inka.de/sites/bigred/devel/tcp-tcp.html.

Best of luck to you!

Regards,
Stephen

[NewtownGuy]

Hi Stephen,

Thank you.

I'm unclear on a few things for setting up the tcp version to run alongside the current udp version:

...1) can both share port 1194 since one is udp and the other is tcp ?

...2) how do I configure the ip addresses that are given to the clients by the tcp server ? I want all tcp and udp clients to be able to communicate and share the current routes set in the ccd folder as though they were still all running on udp. Here is what I've drafted for a portion of server_tcp.conf:

mode server
tls-server
#ifconfig 10.254.0.1 10.254.0.2 -- udp
ifconfig 10.255.2.1 10.255.2.2
#ifconfig-pool 10.254.1.5 10.254.1.251 -- udp
ifconfig-pool 10.254.2.5 10.254.2.251
#route 10.254.0.0 255.255.254.0 -- udp
route 10.254.2.0 255.255.254.0
#push "route 10.254.0.0 255.255.254.0" -- udp
push "route 10.254.2.0 255.255.254.0"

The commented ones are used by the current udp version. Are the uncommented ones correct for the new, tcp version ? In the original udp version, I assign ip addresses in one subnet, 10.254.0.0/24 to clients that are named in ccd, and in the next subnet, 10.254.1.0/24, to clients that are not named in ccd.

--

[NewtownGuy]

To: Stephen

I did two traceroutes from the client machine in California to our two OpenVPN servers in CT that use two different ISP's. The routes diverge almost immediately, on the fourth hop of about 30. I don't know how to determine where in the path the problem is occurring that stops one instance of OpenVPN with UDP in California from connecting.

What should I look for and what other test should I run ? I'd like to be able to predict network problems rather than having to respond to them. Whatever the problem is, it only started occurring recently.

Shouldn't there be some sort of a control in OpenVPN, other than using tcp, that enables it to handle such network problems ?

--

[gladiatr72]

Hello.

To answer your question about tcp/udp ports: UDP port 1194 and TCP port 1194 are different ports. You can think of it in terms of street addresses: 1500 8th Ave. isn't the same destination as 1500 23rd Ave. In your case, you can successfully bind separate processes to each of them.

You shouldn't have any problems with utilizing the same ccd directory for both udp and tcp clients. OpenVPN will intelligently *not* try to push a route that the client has a local destination for.

As far as the connectivity between your office and your data center, you might want to check out http://www.bitwizard.nl/mtr/ There are packages for ubuntu and redhat--regardless of which distro you're running, you should be able to find it in package form. Just run it from the command line: mtr [troublesome remote IP]. Let it run for a few minutes and you'll start to see which part of your network path is being problematic. You might also consider running an mtr session on your remote VPN server as well.

-Stephen

[NewtownGuy]

To: Stephen

I now have one client and a server using tcp. They are able to communicate with one another as shown in their log files, but I can't ping between them. I think there is something wrong with the way that I setup the IP addresses in the server since the ccd for the tcp client pushes an ip address of 10.254.0.33 but the tcp server is on 10.254.2.0/24.

I am confused as to how to set up the ip addresses in the tcp server so as to be compatible with the ones used in the udp server on the same machine.

I have this for the udp server:

mode server
tls-server
ifconfig 10.254.0.1 10.254.0.2
ifconfig-pool 10.254.1.5 10.254.1.251
route 10.254.0.0 255.255.254.0
push "route 10.254.0.0 255.255.254.0"

And this on the tcp server:

mode server
tls-server
#ifconfig 10.254.0.1 10.254.0.2
ifconfig 10.255.2.1 10.255.2.2
#ifconfig-pool 10.254.1.5 10.254.1.251
ifconfig-pool 10.254.2.5 10.254.2.251
#route 10.254.0.0 255.255.254.0
route 10.254.2.0 255.255.254.0
#push "route 10.254.0.0 255.255.254.0"
push "route 10.254.2.0 255.255.254.0"

Do I have the correct values for the tcp server ?

--

[NewtownGuy]

I changed the configuration of the tcp server because there was a typo (a 255 that should have been a 254). Now I get this error message, which I think says I have a routing problem. The client can ping the server, but the server can't ping the client.

On the server, I get this (note the status=1 error):

Mon Jan 17 17:47:54 2011 us=120032 client/a.b.c.d:59011 PUSH: Received control message: 'PUSH_REQUEST'
Mon Jan 17 17:47:54 2011 us=120298 client/a.b.c.d:59011 SENT CONTROL [client]: 'PUSH_REPLY,route 10.254.0.0 255.255.254.0,ping 10,ping-restart 120,ifconfig 10.254.0.33 10.254.0.34' (status=1)

I now have these commands to set the routes in the udp server:

mode server
tls-server
ifconfig 10.254.0.1 10.254.0.2
ifconfig-pool 10.254.1.5 10.254.1.251
route 10.254.0.0 255.255.254.0
push "route 10.254.0.0 255.255.254.0"

And these commands to set the routes in the tcp server:

mode server
tls-server
#ifconfig 10.254.0.1 10.254.0.2
ifconfig 10.254.2.1 10.254.2.2
#ifconfig-pool 10.254.1.5 10.254.1.251
ifconfig-pool 10.254.2.5 10.254.2.251
#route 10.254.0.0 255.255.254.0
route 10.254.0.0 255.255.254.0
#push "route 10.254.0.0 255.255.254.0"
push "route 10.254.0.0 255.255.254.0"

I have these routes on the server:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.254.2.2 * 255.255.255.255 UH 0 0 0 tun0
192.168.192.0 * 255.255.255.0 U 0 0 0 eth0
10.254.0.0 10.254.2.2 255.255.254.0 UG 0 0 0 tun0
default 192.168.192.1 0.0.0.0 UG 100 0 0 eth0

What am I doing wrong ?

--

[gladiatr72]

I'm not seeing a route being pushed to your tcp clients for the client subnet of the udp server (10.254.2/24)

You can verify your configuration by checking the routing table on your clients.

-S

[NewtownGuy]

Hi Gladiatr72,

Thank you for your input. I'm still doing something wrong.

Here's what I'm trying to do: I have a pool of ip addresses, 10.254.0.0/23 that I want clients to be able to access regardless of whether they were activated by the udp client (10.254.0.1) or the tcp client (10.254.2.1). Clients get assigned a static ip address using ccd.

Here's the ccd for a particular client, so it must be getting routing information from the tcp server for the 254 network:

ifconfig-push 10.254.0.101 10.254.0.102

Here's the routing table for that client:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.254.0.102 * 255.255.255.255 UH 0 0 0 tun2
10.210.0.102 * 255.255.255.255 UH 0 0 0 tun1
10.211.0.102 * 255.255.255.255 UH 0 0 0 tun0
192.168.13.0 * 255.255.255.0 U 0 0 0 eth0
10.210.0.0 10.210.0.102 255.255.254.0 UG 0 0 0 tun1
10.211.0.0 10.211.0.102 255.255.254.0 UG 0 0 0 tun0
10.254.0.0 10.254.0.102 255.255.254.0 UG 0 0 0 tun2
default 192.168.13.31 0.0.0.0 UG 100 0 0 eth0

I have three VPN's. It's the 254 that's giving me trouble.

Here's the routing table on the VPN server that hosts the udp and tcp VPN's for 254:

Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.254.2.2 * 255.255.255.255 UH 0 0 0 tun0
10.254.0.2 * 255.255.255.255 UH 0 0 0 tun1
192.168.192.0 * 255.255.255.0 U 0 0 0 eth0
10.254.0.0 10.254.0.2 255.255.254.0 UG 0 0 0 tun1
10.254.0.0 10.254.2.2 255.255.252.0 UG 0 0 0 tun0
default 192.168.192.1 0.0.0.0 UG 100 0 0 eth0

Here's the latest code for the tcp server:

mode server
tls-server
#ifconfig 10.254.0.1 10.254.0.2
ifconfig 10.254.2.1 10.254.2.2
#ifconfig-pool 10.254.1.5 10.254.1.251
ifconfig-pool 10.254.2.5 10.254.2.251
#route 10.254.0.0 255.255.254.0
route 10.254.0.0 255.255.252.0
#push "route 10.254.0.0 255.255.254.0"
push "route 10.254.0.0 255.255.254.0"
push "route 10.254.2.0 255.255.252.0"

The client times out when it tries to ping 10.254.0.1 and 10.254.2.1. The server reports an unknown host when it tries to ping 10.254.0.101.

Can you tell me what my error is ? Thank you in advance.

--

[g4m3c4ck]

Do you have to run the UDP and TCP openvpn server on different networks? I would think that since they are different protocols you would not have a problem binding them to the same port