Increase transfer speeds on network with single thread

[dolphs]

[Dear experts,

ovpn 2.4.1 has been installed between two homes ( both on N3150NH hardware and debian/Jessie with AESNI enabled ).
- ovpn01, upload 30 Mbit/ download 100Mbit- VDSL2
- ovpn02, upload 150Mbit/ download 300Mbit- fiber

As it seems from my good old dm8000 to vuuno4k, there is just enough bandwidth to stream HD (20Mbit).
Unfortunately the other way around there is only 10Mbit available, while 15Mbit is the minimum for this exercise.
- ovpn02 (dm8000) -> ovpn01 (vuuno4k): FAIR ( 20Mbit while >80 should be possible using multiple threads )
- ovpn01 (vuuno4k)-> ovpn02 (dm8000) : NOK! ( 10Mbit while >20 should be possible using multiple threads )

If connection is established between both ovpn servers directly it shows:
- ovpn02 -> ovpn01 : FAIR ( 40Mbit, max download VDSL2 line 100Mbit though )
- ovpn01 -> ovpn02 : OK ( 27Mbit, max upload VDSL2 line )

My goal is both ends should have 15-20Mbit continuously which should be sufficient for (U)HD streams to a 50" TV.
below ( could not find attachment button ) iperf data, which show results from single and 3/5 parallel threads.

Seeking help hoping to increase transfer speed, thus all bandwidth when using single thread.
Also seeking advice rgd:
- Whether Hyperthreading in BIOS should be switched off ?
- configure TCP in ovpn instead of UDP considering HD-video (http) streams ?
+ Could it be somehow using UDP instead of TCP results in 10mbps throughputs ?
- tuning needed on ovpn linux kernel as well kernel on stbs ( dm8000/ vuuno4k ) ?


Please note on server.conf following parameters have been added:

proto udp4

sndbuf 393216
rcvbuf 393216
push "sndbuf 393216"
push "rcvbuf 393216"

(also added to client.conf )
comp-lzo no
fast-io



stream from vuuno4k to dm8000
---
root@dm8000:~# iperf -s
root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t60 -P5

------------------------------------------------------------
[ 7] local 192.168.10.16 port 55916 connected with 192.168.20.12 port 5001
[ 3] local 192.168.10.16 port 55913 connected with 192.168.20.12 port 5001
[ 5] local 192.168.10.16 port 55914 connected with 192.168.20.12 port 5001
[ 6] local 192.168.10.16 port 55915 connected with 192.168.20.12 port 5001
[ 4] local 192.168.10.16 port 55912 connected with 192.168.20.12 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-60.0 sec 44.9 MBytes 6.27 Mbits/sec
[ 6] 0.0-60.1 sec 35.2 MBytes 4.92 Mbits/sec
[ 7] 0.0-60.1 sec 40.0 MBytes 5.58 Mbits/sec
[ 3] 0.0-60.2 sec 35.8 MBytes 4.98 Mbits/sec
[ 4] 0.0-60.2 sec 42.4 MBytes 5.91 Mbits/sec
[SUM] 0.0-60.2 sec 198 MBytes 27.6 Mbits/sec



root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t60 -P3
------------------------------------------------------------
[ 5] local 192.168.10.16 port 55922 connected with 192.168.20.12 port 5001
[ 3] local 192.168.10.16 port 55921 connected with 192.168.20.12 port 5001
[ 4] local 192.168.10.16 port 55920 connected with 192.168.20.12 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-60.1 sec 67.0 MBytes 9.36 Mbits/sec
[ 5] 0.0-60.1 sec 67.0 MBytes 9.35 Mbits/sec
[ 3] 0.0-60.1 sec 64.4 MBytes 8.99 Mbits/sec
[SUM] 0.0-60.1 sec 198 MBytes 27.7 Mbits/sec



### Below clearly shows a single thread does not utilise the full 27Mbit upload spead
root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.10.16 port 55917 connected with 192.168.20.12 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.1 sec 71.9 MBytes 10.0 Mbits/sec

------------------------------------------------------------
[ 3] local 192.168.10.16 port 55931 connected with 192.168.20.12 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.1 sec 70.0 MBytes 9.77 Mbits/sec


### to be certain, trace back:
###
### Vu+ to ovpn server locally
###
### OK

root@vuuno4k:/# iperf -c 192.168.10.11 -p5001 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.10.16 port 51065 connected with 192.168.10.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 6.57 GBytes 941 Mbits/sec



### Vu+ to router locally
###
### FAIR, would have expected similar as to .11

root@vuuno4k:/# iperf -c 192.168.10.10 -p4711 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.10.16 port 53330 connected with 192.168.10.10 port 4711
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 1.09 GBytes 156 Mbits/sec



### Vu+ to router remotely
###
### NOK, only 10Mbit ?

root@vuuno4k:/# iperf -c 192.168.20.10 -p4711 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.10.16 port 39514 connected with 192.168.20.10 port 4711
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.1 sec 70.2 MBytes 9.81 Mbits/sec



### Vu+ to ovpn server remotely:
###
### OK, as upload limited to 28Mbit

root@vuuno4k:~# iperf -c 192.168.20.11 -p5001 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.10.16 port 58933 connected with 192.168.20.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 177 MBytes 24.8 Mbits/sec


### then from ovpn server to dm8000 :
###
### OK, as dm800 has 100Mbit LAN

root@ovpn02:~# iperf -c 192.168.20.12 -p5001 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.20.11 port 47388 connected with 192.168.20.12 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 666 MBytes 93.0 Mbits/sec




stream from dm8000 to vuuno4k
---
root@vuuno4k:~# iperf -s
root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t60 -P5
------------------------------------------------------------
[ 7] local 192.168.20.12 port 32897 connected with 192.168.10.16 port 5001
[ 4] local 192.168.20.12 port 32894 connected with 192.168.10.16 port 5001
[ 3] local 192.168.20.12 port 32893 connected with 192.168.10.16 port 5001
[ 5] local 192.168.20.12 port 32895 connected with 192.168.10.16 port 5001
[ 6] local 192.168.20.12 port 32896 connected with 192.168.10.16 port 5001
[ ID] Interval Transfer Bandwidth
[ 7] 0.0-60.0 sec 133 MBytes 18.6 Mbits/sec
[ 4] 0.0-60.0 sec 133 MBytes 18.6 Mbits/sec
[ 6] 0.0-60.0 sec 133 MBytes 18.6 Mbits/sec
[ 3] 0.0-60.0 sec 133 MBytes 18.6 Mbits/sec
[ 5] 0.0-60.0 sec 133 MBytes 18.6 Mbits/sec
[SUM] 0.0-60.0 sec 666 MBytes 93.0 Mbits/sec



root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t60 -P3
------------------------------------------------------------
[ 5] local 192.168.20.12 port 32901 connected with 192.168.10.16 port 5001
[ 3] local 192.168.20.12 port 32899 connected with 192.168.10.16 port 5001
[ 4] local 192.168.20.12 port 32900 connected with 192.168.10.16 port 5001
[ ID] Interval Transfer Bandwidth
[ 5] 0.0-60.0 sec 154 MBytes 21.6 Mbits/sec
[ 3] 0.0-60.0 sec 154 MBytes 21.5 Mbits/sec
[ 4] 0.0-60.0 sec 143 MBytes 20.0 Mbits/sec
[SUM] 0.0-60.0 sec 451 MBytes 63.1 Mbits/sec



root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t60 -P1
------------------------------------------------------------
[ 3] local 192.168.20.12 port 32903 connected with 192.168.10.16 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 145 MBytes 20.3 Mbits/sec






root@ovpn02:~# iperf -c 192.168.10.11 -p5001 -t60 -P1
------------------------------------------------------------
Client connecting to 192.168.10.11, TCP port 5001
TCP window size: 45.0 KByte (default)
------------------------------------------------------------
[ 3] local 10.8.0.2 port 46666 connected with 192.168.10.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.2 sec 290 MBytes 40.5 Mbits/sec

( theoretically 100Mbit can be obtained ( due to download limit VDSL2 line )


root@ovpn01:~# iperf -c 192.168.20.11 -p5001 -t60 -P1

------------------------------------------------------------
[ 3] local 10.8.0.1 port 57258 connected with 192.168.20.11 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-60.0 sec 197 MBytes 27.5 Mbits/sec

( max upload ! )

[dolphs]

meanwhile I dug a little further.
Came to my attention the TCP window sizes differ, but as I am not an expert in this field I like to find out if that is a point of concern.
imho these should be equal between both ends to gain the best performance ...

1/
vuuno4k streams to dm8000, INSUFFICIENT and higher throughput expected ( P1 shows 10Mbit, while P3 shows max speed )
---
vuuno4k [192.168.10.16] -> dm8000 [ 192.168.20.12 ] => 10Mbit [ NOK! with P1 ] / [ OK, 27Mbit with P3 ]
vuuno4k [192.168.10.16] -> ovpn_remote [ 192.168.20.11 ] => 27Mbit [ OK, while upload VDSL2 line max 30Mbit ]
---
ovpn [192.168.20.11] -> dm8000 [ 192.168.20.12 ] => 92Mbit [ OK, while eth0 dm8000 100Mbit ]
# both ovpn and dm8000 are on same switch ( Netgear GS105Ev2 – 5-Port Gigabit ProSAFE Plus Switch )


2/
dm8000 streams to vuuno4k, FAIR but higher throughput expected ( P1 shows only 22Mbit, while P5 shows max speed )
---
dm8000 [192.168.20.12] -> vuuno4k [ 192.168.10.16 ] => 22Mbit [ FAIR ]
dm8000 [192.168.20.12] -> ovpn_remote [ 192.168.10.11 ] => 22Mbit [ FAIR ]
---
ovpn [192.168.10.11] -> vuuno4k [ 192.168.10.16 ] => 941 Mbits/sec [ OK, while eth0 vuuno4k 1000Mbit ]
# ovpn on VDSL2-router, vuuno4k behind switch ( Netgear GS105Ev2 – 5-Port Gigabit ProSAFE Plus Switch )


rgd 1/
Aiming for at least 20Mbit ( while 30Mbit is max due to VDSL2 upload limit ) between [ 192.168.10.16 ] -> [ 192.168.20.12 ]

TCP window size: 85,3 KByte while clients connect only with 43.8 KByte ?
Is it needed to set these equally on both ends?


root@dm8000:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t10 -P1
------------------------------------------------------------
Client connecting to 192.168.20.12, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------



rgd 2/
Aiming for at least 70Mbit ( while 100Mbit is max due to VDSL2 download limit ) between [ 192.168.10.12 ] -> [ 192.168.20.16 ]


TCP window size: 85,3 KByte while clients connect only with 19.7 KByte ?!
Same question is it needed to set these equally on both ends?


root@vuuno4k:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------

root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t10 -P1
------------------------------------------------------------
Client connecting to 192.168.10.16, TCP port 5001
TCP window size: 19.7 KByte (default)
------------------------------------------------------------

[dolphs]

looks like I solved the issue.
Mainly from other posts all over the place, to summarise

on ovpn servers:
- updated txqueuelen from 100 to 1000 on tun0 ( as 1000 is also on eth0 )
- added random number seeder to increase entropy
- updated congestion control from cubic to htcp
- increase Linux autotuning TCP buffer limit to 32MB ( tcp_rmem / tcp_wmem - 4096 87380 33554432 )
- updated max_back log from 1000 to 5000

on stbs:
increased tcp_rmem / tcp_wmem and rmem_max / wmem_max



root@vuuno4k:~# iperf -c 192.168.20.12 -p5001 -t60 -P1
 
root@dm8000:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 42769
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.1 sec   160 MBytes  22.4 Mbits/sec
[  5] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 54890
[  5]  0.0-60.5 sec   193 MBytes  26.8 Mbits/sec
[  4] local 192.168.20.12 port 5001 connected with 192.168.20.11 port 42785
[  4]  0.0-60.0 sec   171 MBytes  23.9 Mbits/sec
 
 
 
root@dm8000:~# iperf -c 192.168.10.16 -p5001 -t60 -P1
 
root@vuuno4k:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51269
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-60.2 sec   670 MBytes  93.3 Mbits/sec
[  5] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51270
[  5]  0.0-60.2 sec   654 MBytes  91.1 Mbits/sec
[  4] local 192.168.10.16 port 5001 connected with 192.168.20.12 port 51271
[  4]  0.0-60.2 sec   626 MBytes  87.3 Mbits/sec