Openvpn(tap, udp), LTE, tunnel collapses during high load

[derpert]

Hi,

sorry if this is an inappropriate place to ask because i am using openvpn within Zeroshell.
However it is very convenient and i would like to continue using it despite it's terrible GUI and Bugs.

On the server side i am using a Ramnode NL 256MB SKVM Server running Zeroshell and an openvpn(udp, tap) Tunnel.

On the client side i am using another Zeroshell Box with an LTE Connection(Hutchison Austria).
http://www.speedtest.net/result/3597852170.png

However as soon as i am approaching bandwith levels of around ~40 Mbps my tunnel collapses. I have no such problems using tcp but the performance is terrible.

The log(client) i get within Zeroshell:

Code: Select all

Inactivity timeout (--ping-restart), restarting
/root/kerbynet.cgi/scripts/vpn_mii VPN03 1500 1560 init
SIGUSR1[soft,ping-restart] received, process restarting

After the client disconnected the server still thinks the tunnel is online and the client is henceforth unable to reconnect but that seems to be one of the many Zeroshell Bugs.

I tried various --tun-mtu, --fragment, --mssfix values but it always stayed the same.

With "--txqueuelen 50" i get a stable tunnel but i cannot fully saturate my upstream.

Does anybody perhaps have any ideas or experiences to share using LTE connections running openvpn(udp, tap)? Perhaps this is somehow an LTE problem, i don't know. Never had such problems using cable. Since this is a private line my ISP will probably be of little help.

[derpert]

Thank you,

i don't know where Zeroshell stores it's configuration and their forums are kinda abandoned. Verb 4 log level did not really get me much using openvpn in Zeroshell.

I am not bridging yet. The timeout happens during high load so i don't see that doing much but i will try your options later.
Right now, after reading over my topic i thought that perhaps two tunnels might solve my problem and indeed it did by bonding two tunnels together.
Now i can saturate my connection and the tunnels do not disconnect during high load.

When i have time i will configure openvpn in debian manually and see if the results are the same. Then i should be able to provide sufficient logs as well.

[derpert]

Regarding my last post, when bonding together two tunnels it kinda works. I can saturate it to the brink but the upstream is suddenly abysmal and single instances of a Download are much slower as well.

Anyhow, i just tried to run openvpn manually, using Debian, on the client and the server.

When i use udp and tun i can connect and ping but as soon as i want try to use http it just sits there waiting for a response. This is the same behaviour as with vyprvpn on openvpn, they use udp and tun as well. However if i use HSDPA instead of LTE it instantly works.

Code: Select all

wget http://rbx.proof.ovh.net/files/1Gio.dat
--2014-06-29 09:45:10--  http://rbx.proof.ovh.net/files/1Gio.dat
Resolving rbx.proof.ovh.net (rbx.proof.ovh.net)... 188.165.12.106, 2001:41d0:2:876a::1
Connecting to rbx.proof.ovh.net (rbx.proof.ovh.net)|188.165.12.106|:80... connected.
HTTP request sent, awaiting response...

Now, when i try to use udp and tap everything works as described before. I can connect and everything is just fine as long as i don't generate a lot of traffic but when i do the tunnel becomes unusable and is unable to reconnect in almost all cases i tried.

This is the simple configuration i used for tap during my test. Both server and client where running on vServers. Which i am starting to think might be a possible source for my problems? That or the LTE connection. My next step is probably to test if the same behaviour occurs on dedicated machines for both the server and the client.

I also noticed that the "--shaper bytes_per_second" option does not seem to work?

Here is a wireshark log showing the openvpn Traffic:
https://www.dropbox.com/s/dq8vdaq0a3kr5 ... oad.pcapng

In the wireshark log i get a lot of "MessageType: Unknown Messagetype[Malformed Packet]" which i assume are not normal?

// Server
openvpn --remote 178.113.17.113 --dev tap0 --ifconfig 10.9.8.1 255.255.255.0 --keepalive 1 60 --verb 4 --log server.log
https://www.dropbox.com/s/4v9t0tz57erq1z9/server.log

// Client
openvpn --remote 107.161.29.95 --dev tap0 --ifconfig 10.9.8.2 255.255.255.0 --keepalive 1 60 --verb 4 --log client.log
https://www.dropbox.com/s/aergqmsldb0nfii/client.log

/sbin/route add -net 188.165.12.106 netmask 255.255.255.255 gw 10.9.8.1 <= this is the ip rbx.proof.ovh.net resolves to

Then i start a couple of Downloads over the tunnel and the tunnel becomes unusable and kinda stays in limbo without reconnecting.
wget http://rbx.proof.ovh.net/files/1Gio.dat -b

I was running a ping to the server during my test:
64 bytes from 10.9.8.1: icmp_req=6486 ttl=64 time=212 ms
64 bytes from 10.9.8.1: icmp_req=6487 ttl=64 time=177 ms
64 bytes from 10.9.8.1: icmp_req=6488 ttl=64 time=177 ms
64 bytes from 10.9.8.1: icmp_req=6489 ttl=64 time=187 ms
64 bytes from 10.9.8.1: icmp_req=6490 ttl=64 time=203 ms
From 10.9.8.2 icmp_seq=6525 Destination Host Unreachable
From 10.9.8.2 icmp_seq=6526 Destination Host Unreachable
From 10.9.8.2 icmp_seq=6527 Destination Host Unreachable
From 10.9.8.2 icmp_seq=6528 Destination Host Unreachable
From 10.9.8.2 icmp_seq=6529 Destination Host Unreachable

[derpert]

Try this:

# Server:

Code: Select all

openvpn --ifconfig 10.88.0.221 255.255.255.252 -verb 1 --log server.log --ping 60 --ping-restart 600 --dev tap

# Client:

Code: Select all

openvpn --ifconfig 10.88.0.222 255.255.255.252 -verb 1 --log client.log --ping 60 --ping-restart 600 --den tap --remote {your choice}

I can connect and ping the server from the client and ping the client from the server but when i try to send traffic over the tunnel i get no response(neither ping nor http get).

Logs:
https://www.dropbox.com/s/qwuccqdtd7579cy/client.log
https://www.dropbox.com/s/48p9vbw9jl559zj/server.log

[derpert]

Ah,

i forgot to enable NAT. Now i can ping/initiate Downloads(ovh) however the initial problem still remains. After the tunnel gets saturated the tunnel suddenly stops receiving data. However if i shape the tunnel i have a perfectly stable tunnel.

Code: Select all

tap0      Link encap:Ethernet  HWaddr ee:1c:0d:09:96:4f  
          inet addr:10.88.0.222  Bcast:10.88.0.223  Mask:255.255.255.252
          inet6 addr: fe80::ec1c:dff:fe09:964f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:36590 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19020 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:52303578 (49.8 MiB)  TX bytes:1256430 (1.1 MiB)

Logs:
https://www.dropbox.com/s/ynbe8gcg44q8npu/client.log
https://www.dropbox.com/s/mvv8425xpsmvyfu/server.log

[derpert]

When looking at the tap traffic during downloads i am seeing a lot of tcp dup acks.

http://i.imgur.com/Kp99Xa4.png

[derpert]

The tunnel collapse even happens during one fast download(no tcp dup acks to be seen). It goes up to roughly 5.4 MB/second and then just suddenly stops. I attached a tcpdump of the tap device during the download. It looks like everything is in order up to the point where the transfer suddenly stops...

I am able to saturate my connection using two tunnels bonded together without any disconnects as long both tunnels stay below this threshold.
So i guess either the KVM Server is the problem or the ISP?

tcpdump of the tap device on the client during a single download:
https://www.dropbox.com/s/xrz55w66f0t65 ... r_vpn.pcap

[derpert]

Any ideas left?

[derpert]

I thought i would update this thread so we can put this to bed. As it turned out there was some problem with the Equipment my ISP used but after a valiant effort shown by one of Hutchison Austrias technicians this problem was finally solved.