Can`t brake 200Mbs limit (Linux/Windows).

[ArthurX]

Good day!

Last week I spent many hours to breake OpenVPN bandwidth limit 200Mbs.
I use two different network topology and OS, but result same - 200Mbs.

First topology:
Ubuntu16(client X5645)<------>ISP(500Mbs)<----->FW<----->Ubuntu16(server X5670)
OpenVPN 2.3.10 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6]
OpenSSL 1.0.2g 1 Mar 2016, LZO 2.08



Server:

Code: Select all

port 1195
proto udp
dev tap0

ca ca.crt
cert server.crt
key server.key
dh dh1024.pem

sndbuf 1572864
rcvbuf 1572864
push "sndbuf 1572864"
push "rcvbuf 1572864"

server 172.16.8.0 255.255.255.128
client-config-dir ccd
client-to-client

keepalive 10 120

tcp-nodelay
fast-io

cipher AES-128-CBC
auth none

comp-lzo
push "comp-lzo yes"

Client:

Code: Select all

client
dev tap0
proto udp

remote x.x.x.x 1195
resolv-retry infinite

nobind

ca ca.crt
cert client.crt
key client.key

remote-cert-tls server
cipher AES-128-CBC
auth none
pull

And second topology:
Laptop(corei5_Win7)<------------------>Laptop(corei5_Win10)
OpenVPN 2.3.8 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Aug 4 2015
OpenSSL 1.0.1p 9 Jul 2015, LZO 2.08



Server:

Code: Select all

port 1195
proto udp
dev tap

ca ca.crt
cert server.crt
key server.key
dh dh1024.pem

server 172.16.8.0 255.255.255.128
client-config-dir ccd

sndbuf 3145728
rcvbuf 3145728
push "sndbuf 3145728"
push "rcvbuf 3145728"

cipher AES-128-CBC
auth none
comp-lzo yes
push "comp-lzo yes"

Client:

Code: Select all

client
dev tap
proto udp
remote 10.10.10.1 1195

resolv-retry infinite
nobind

ca ca.crt
cert client.crt
key client.key
remote-cert-tls server

cipher AES-128-CBC
auth none
pull

Even between to laptops I can`t reach more than 230Mbs (average 200mbs). Many tests with different values such as tun-mtu, mssfix, fragment etc. does not any effect. Only sndbuf/rcvbuf 3MB values help`s reach 200Mbs.

Help, pls!

[Pippin]

Hi,

Try dev tun, both sides.

And up/down Speed from ISP?

And take a look here:
https://community.openvpn.net/openvpn/w ... orks_Linux

[ArthurX]

Today, end of the day, I tried download 2 different files from 2 different servers in LAN and throughput dramatically increase two times. Now I have 400Mbps average with 500Mbps peaks!



I use bonding with next parameters:

Code: Select all

# Tap interfaces
auto tap0
iface tap0 inet manual
pre-up openvpn --mktun --dev tap0
post-down openvpn --rmtun --dev tap0

auto tap1
iface tap1 inet manual
pre-up openvpn --mktun --dev tap1
post-down openvpn --rmtun --dev tap1

# Bonding interfaces
auto bond0
iface bond0 inet static
 address 172.16.9.1
 netmask 255.255.255.252
 bond-slaves none
 bond-mode 802.3ad
 bond_xmit_hash_policy layer3+4
# bond-mode balance-rr
 bond-miimon 100
 bond-downdelay 200
 bond-updelay 200

My conclusion about 200Mbts throughput limit in Linux:
1. Xeon X5645 (client) can`t do more. I doubt about that.
2. Need additional OS configuration or change OS to FreeBSD or other.
3. Change bond mode to ... round-robin maybe.

About Windows, doesn`t matter, I did test in Windows just for fun.

[ArthurX]

Hello!

How to bind each openvpn process to specific cpu?
I`m pretty sure - need edit init script, but I think just adding string "numactl --physcpubind=1 " not enought. I have 2 process, so 1st process need bind to 1st cpu, 2nd process to 2nd cpu.
Need little help!

[ArthurX]

I think "taskset" easy way, using this tool now.
for example:
top
1028 /usr/sbin/openvpn --daemon ovpn-client1
1092 /usr/sbin/openvpn --daemon ovpn-client2

taskset -p 01 1028
taskset -p 02 1092

Work fine!