openwrt based devices - speed limited

bolvan

I have 2 openwrt based devices. One of them is based on MIPS 650 mhz single core, another ARM 1Ghz 2 core.
Both have external IP addresses. Iperf shows 33 mbps between ipv4 endpoints.
But I get only ~6 mbps in openvpn tunnel. CPU is loaded 30 .. 50%, some sirq, but always have ~30% idle on both ends.
Openvpn is configured for UDP. Tried all known speed tweaks without success.

server :

fast-io
persist-key
persist-tun
tun-ipv6
ca /etc/ca.pem
cert /etc/cert.pem
cipher AES-128-CBC
client-config-dir ccd
comp-lzo no
dev tun1
dh /etc/openvpn/dh.pem
keepalive 10 30
key /etc/key.pem
mssfix 1330
port 5
proto udp
reneg-sec 86400
resolv-retry infinite
server 192.168.254.32 255.255.255.224
tls-cipher AES128+EECDH:AES128+EDH
topology p2p
tun-mtu 1370
verb 3
push route 192.168.254.32 255.255.255.224
push route 192.168.2.0 255.255.254.0
push route-ipv6 fdff:ffff:ffff::/64
push route-ipv6 fd15::/48
server-ipv6 fdff:ffff:ffff::/64

client :

client
fast-io
persist-key
persist-tun
tun-ipv6
ca /etc/ca.pem
cert /etc/cert.pem
cipher AES-128-CBC
comp-lzo no
dev tun1
keepalive 10 30
key /etc/key.pem
mssfix 1330
port 5
proto udp
remote my.server.name 5
reneg-sec 86400
resolv-retry infinite
tls-cipher AES128+EECDH:AES128+EDH
tun-mtu 1370
verb 3
verify-x509-name CN=vpn.cool.domain

Traffic

Tunnel speed is limited to the lowest upload speed of your end points.

bolvan

Traffic wroteTunnel speed is limited to the lowest upload speed of your end points.

I should be more technical

endpoint1 => endpoint2
[ 5] 0.0-10.0 sec 58.8 MBytes 49.3 Mbits/sec
endpoint2 => endpoint1
[ 3] 0.0-10.0 sec 57.0 MBytes 47.9 Mbits/sec
tun. smb copy file
[ 3] 0.0-10.1 sec 7.00 MBytes 5.83 Mbits/sec

cpu load endpoint 1

CPU:  17% usr   8% sys   0% nic  45% idle   0% io   0% irq  28% sirq
Load average: 0.09 0.05 0.05 3/55 16056
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
10566  5151 root     S     3596   6%  41% /usr/sbin/openvpn --syslog openvpn(tu
    3     2 root     SW       0   0%   4% [ksoftirqd/0]

cpu load endpoint 2

Mem: 1007428K used, 22944K free, 17892K shrd, 2012K buff, 838080K cached
CPU:   9% usr   7% sys  10% nic  33% idle   0% io   0% irq  38% sirq
Load average: 0.58 0.46 0.34 2/80 24231
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
23897  4948 root     S     3360   0%  26% /usr/sbin/openvpn --syslog openvpn(tu

Traffic

bolvan wroteI should be more technical

Even more so ..

OpenVPN basically functions at whatever speed the underlying equipment can support.
This includes all the network links between your devices and outside of your control.

bolvan

Traffic wrote
bolvan wroteI should be more technical
Even more so ..

OpenVPN basically functions at whatever speed the underlying equipment can support.
This includes all the network links between your devices and outside of your control.

How do you explain that iperf can and openvpn cant ?

Traffic

OpenVPN is fully encrypted .. iperf is plain text .. there is an overhead for encryption.

Exactly what are your ISP advertised speeds and what does speedtest.net show ?

bolvan

Traffic wroteOpenVPN is fully encrypted .. iperf is plain text .. there is an overhead for encryption.

Exactly what are your ISP advertised speeds and what does speedtest.net show ?

I tried setting cipher to none. It increased speed only by 150 kb /sec. CPU is not bottleneck as 'top' shows.
I posted here results from iperf between endpoints. This test is more accurate than speedtest because it show actual speed between involved IPs and not to arbitrary location. Anyway speedtest shows 45/38 mbits at the moment. Another endpoint is even faster.
switching openvpn to tcp gives almost the same speed
tested that no IP fragmentation occurs
recv, send buffer sizes, tun queue length change nothing