Page 1 of 2
Very slow transfer speeds
Posted: Thu Nov 03, 2016 10:22 pm
by dancos
Hello OpenVPN community,
I've been struggling with this for the past days and I have not made any progress so far. The configuration of OpenVPN was delightfully straight forward and pretty much everything worked out of the box. The only thing wrong with the installation are the transfer speeds. Without OpenVPN "iperf" reports about 300 mbps download speed. As soon as OpenVPN is involved the speed drops to anywhere between 10 and 70 mbps depending on different mtu/crypto configurations.
I've been able to reproduce the issue in a simple adhoc VPN.
Server (Debian 8):
Code: Select all
openvpn --dev tun --proto udp --port 11000 --secret static.key --ifconfig 192.168.222.11 192.168.222.10
iperf -c 192.168.222.10
Client (Ubuntu 16.04):
Code: Select all
openvpn --dev tun --proto udp --port 11000 --secret static.key --ifconfig 192.168.222.10 192.168.222.11 --remote server.ip
iperf -s
This gives me about 35 mbps on a link that should be 300 mbps. The situation is much worse when using tcp at 2-11 mbps. I've experimented with countless combinations of tun-mtu/fragment/mssfix/sndbuf/rcvbuf and many more, the maximum I've ever reached was about 60-70 mbps by adding the following parameters:
Code: Select all
--txqueuelen 15000 --tcp-queue-limit 128 --cipher aes-256-cbc
CPU usage on the client is <50% and on the server it's <20%. I've tried different ports as well (443 for example) and iperf over an SSH tunnel gives me almost the full 300 mbps.
While researching this issue I've found people who have the same problem with no solution, people who had to change a tun-mtu or buf setting and the issue was gone, people who say that this is normal and people who say that the overhead should be max 10%. None of the setting changes I've found have helped.
If anyone has any idea as to what might be causing the slowdown and how we can get at least 100mbps out of the VPN that would be greatly appreciated!
Re: Very slow transfer speeds
Posted: Thu Nov 03, 2016 11:16 pm
by TinCanTech
Re: Very slow transfer speeds
Posted: Fri Nov 04, 2016 5:39 am
by dancos
Thank you for your reply. That Wiki entry has helped me run this one-line VPN setup for testing as well as the "--cipher aes-256-cbc" parameter which gave some improvement but the speed remains 70mbps or lower. Changes to mtu values generally made the speed worse, no matter the value.
Even with encryption completely disabled (--cipher none --auth none) the speed remains below 70 mbps (and below 11 mbps using tcp). Outside of the VPN the connection to the same box is still at 300 mbps.
Re: Very slow transfer speeds
Posted: Sat Nov 05, 2016 6:23 pm
by dancos
Maybe as a general question, can this behavior be reproduced by anyone? I.e. using the commands mentioned, can you get almost full link speed?
Searching for a cause for normal behavior would be rather pointless

Re: Very slow transfer speeds
Posted: Sat Nov 05, 2016 7:05 pm
by TinCanTech
If you can describe your network in more detail maybe a bottle neck can be identified.
Personally, I get exactly what I pay for out of my internet while using openvpn.
Re: Very slow transfer speeds
Posted: Sat Nov 05, 2016 9:44 pm
by Pippin
In addition to ^^^^
CPU?
CPU supports AES-NI?
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 5:32 am
by dancos
Thank you for your replies. It's a Linux workstation on a gigabit network behind an ASUS RT-AC68U router that connects to a remote server (ping 17-20 ms, 1 gbps bandwidth) through cable internet (400/40 mbps).
Both CPUs support AES-NI, Server:
Intel(R) Xeon(R) CPU E3-1246 v3 @ 3.50GHz
Client:
Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
Even though my upload is vastly slower than the download I get better upload speed: Download I get about 33 mbps (355 mbps without VPN), upload 38 mbps (41 mbps without VPN). The next test is going to be hooking up the workstation directly to the cable modem, I'll post an update if this makes a difference

/Edit: Download is about 36 mbps without the router, so minimal difference.
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 6:19 am
by dancos
Another interesting test, using the "shaper" option on the server, when it's set to 5000000 (40 mbps) the speed stays the same (36 mbps, same for any shaper value above that). When set to 2500000 (20 mbps) the speed still stays the same, so I get 36 mbps with a 20 mbps limit (and OpenVPN says "Output Traffic Shaping initialized at 2500000 bytes per second").
Reducing shaper to 1250000 (10 mbps) finally limits the bandwidth and I get 10 mbps.
Version information, Server:
OpenVPN 2.3.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Nov 12 2015
library versions: OpenSSL 1.0.1t 3 May 2016, LZO 2.08
Client:
OpenVPN 2.3.10 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Feb 2 2016
library versions: OpenSSL 1.0.2g 1 Mar 2016, LZO 2.08
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 1:00 pm
by TinCanTech
Server Max. upload speed ?
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 1:23 pm
by dancos
Max upload speed of the server would be 1 gbps, 200 mbps guaranteed.
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 1:30 pm
by TinCanTech
Try this on your server and then client:
https://sourceforge.net/speedtest/
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 2:52 pm
by dancos
I believe this is located in the US while the server and client are in Europe. Still the speeds I see are over 200 mbps. Also, iperf between the two gives me 355 mbps server->client and 41 mbps client->server with a ping of about 20 ms.
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 4:59 pm
by Pippin
Asus router has Trend Micro AI?
If so, disable helps?
And yes, test going directly out the "front door"
Re: Very slow transfer speeds
Posted: Sun Nov 06, 2016 9:36 pm
by dancos
Thank you, none of the special functions are enabled. I did test hooking the client directly to the cable modem, it gave me +2-3 mbps but that could also just be variance. Interestingly download seems to be slower than upload even though in terms of bandwidth it should be 10x faster.
Re: Very slow transfer speeds
Posted: Tue Nov 08, 2016 1:32 pm
by Pippin
Sounds like mtu issue.
One time i helped a user having same issue of speed in one direction being bad.
fragment and mssfix did not help any.
Eventually found out that his laptop had the mtu of his network adaptor not standard (1500), it was set to 1300.
Setting it back to default solved the problem.
Re: Very slow transfer speeds
Posted: Wed Nov 09, 2016 10:54 am
by dancos
Thank you for the hint, I've checked all the MTU values for server, tap devices and client (even router, but taking that out of the equation did not make any difference before). It's always 1500, changing the MTU value up/down when starting the tunnel only decreases performance.
It's hard to say whether it's only slow in one direction or whether the tunnel is just somehow stuck at 40 mbps in both directions since that's also my upload limit.
Re: Very slow transfer speeds
Posted: Mon Nov 14, 2016 9:04 am
by dancos
Very odd... I left iperf running now with a BW report every 4 seconds. It looks like I periodically get about 100 mbps for a short time then it's back to 30-40:
[ 3] 88396.0-88400.0 sec 17.0 MBytes 35.7 Mbits/sec
[ 3] 88400.0-88404.0 sec 19.1 MBytes 40.1 Mbits/sec
[ 3] 88404.0-88408.0 sec 49.9 MBytes 105 Mbits/sec
[ 3] 88408.0-88412.0 sec 41.1 MBytes 86.2 Mbits/sec
[ 3] 88412.0-88416.0 sec 20.9 MBytes 43.8 Mbits/sec
... more entries ...
[ 3] 88640.0-88644.0 sec 32.8 MBytes 68.7 Mbits/sec
[ 3] 88644.0-88648.0 sec 17.9 MBytes 37.5 Mbits/sec
[ 3] 88648.0-88652.0 sec 22.8 MBytes 47.7 Mbits/sec
[ 3] 88652.0-88656.0 sec 22.4 MBytes 46.9 Mbits/sec
[ 3] 88656.0-88660.0 sec 26.6 MBytes 55.8 Mbits/sec
[ 3] 88660.0-88664.0 sec 54.5 MBytes 114 Mbits/sec
[ 3] 88664.0-88668.0 sec 17.4 MBytes 36.4 Mbits/sec
[ 3] 88668.0-88672.0 sec 17.8 MBytes 37.2 Mbits/sec
[ 3] 88672.0-88676.0 sec 20.8 MBytes 43.5 Mbits/sec
Does anyone know what could explain such variance?
Re: Very slow transfer speeds
Posted: Thu Dec 22, 2016 5:30 am
by KaszasT
Hello
We have the exact same situation, with worse speeds (iperf 80, 80, 60,40, 20, 0, 0, 0, 20, 60, 80, 40, 0 ,0, etc.). I walked down the same path, nothing helped. Do you got any solution for this problem?
Thanks,
Tibor
Re: Very slow transfer speeds
Posted: Thu Feb 23, 2017 7:56 am
by ludwig.gramberg
having the same problem here, speeds just make no sense
the scenario (all values in mbit down/up):
Code: Select all
client A <---200/10---> vpn-server(100/100) <---300/30---> client B
when client A downloads from client B the speed is limited at precisely 10mbit (avg 9,8)
but that direction should allow for 30mbit since thats the limit here.
then when client B pushed some files to the vpn-server the limit was 20mbit.
doing any up/download outside the realm of openvpn the actual speeds are achieved.
client A and B can easily compute the aes so this should not be the limiting factor, also why does client A not reach at least the 20mbit client B did get? just makes no sense...
I tried the settings which were recommended on the wiki but it did not help.
any advice? also played around with different mtu etc.
server-cfg:
Code: Select all
port 1196
proto tcp
dev tun2
ca ca-*****.crt
cert *****.crt
key ****.key # This file should be kept secret
dh dh2048-****.pem
server 10.10.0.0 255.255.255.0
ifconfig-pool-persist ipp-****.txt
client-config-dir /etc/openvpn/ccd/****
client-to-client
duplicate-cn
keepalive 10 120
cipher AES-256-CBC
comp-lzo no
user nobody
group nogroup
persist-key
persist-tun
status openvpn-status-*****.log
log-append /var/log/openvpn-*******.log
# 0 is silent, except for fatal errors
# 4 is reasonable for general usage
# 5 and 6 can help to debug connection problems
# 9 is extremely verbose
verb 4
mute 10
client-cfg:
Code: Select all
remote ***** 1196 tcp-client
persist-key
tls-client
pull
ca /etc/openvpn/*****/ca.crt
dev tun
persist-tun
cert /etc/openvpn/*****/*****.crt
comp-lzo no
nobind
key /etc/openvpn/*****/*****.key
cipher AES-256-CBC
Re: Very slow transfer speeds
Posted: Thu Feb 23, 2017 11:59 am
by TinCanTech
ludwig.gramberg wrote:speeds just make no sense
ludwig.gramberg wrote:proto tcp
