OpenVPN server seems strangely slow

This forum is for admins who are looking to build or expand their OpenVPN setup.

Moderators: TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech

Forum rules
Please use the [oconf] BB tag for openvpn Configurations. See viewtopic.php?f=30&t=21589 for an example.
houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 9:58 am

Hello,

I wonder if I have a mistake in the server config to have such poor speed. Unless this is expected with the additional security Crypt2 is providing? In this case I have to accept it. But I thought I run it here by you first.

Please check below for comparison:

OpenVPN is installed on a 2019 server with AMD Ryzen™ 5 3600 (6 Cores, 64 GB DDR4)
Image

IKEv2 (StrongSwan) is installed on a 2014 server with Intel Xeon E3-1246V3 (4 Cores, 32 GB DDR3)
Image

Both server are located in the same data centre.


server

port 1789
proto udp
dev tun
user nobody
group nogroup
persist-key
persist-tun
keepalive 10 120
topology subnet
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 1.0.0.1"
push "dhcp-option DNS 1.1.1.1"
push "redirect-gateway def1 bypass-dhcp"
dh none
ecdh-curve prime256v1
tls-crypt-v2 tls-crypt-v2.key
crl-verify crl.pem
ca ca.crt
cert server_7RmgUuwxS0MOJqci.crt
key server_7RmgUuwxS0MOJqci.key
auth SHA256
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
tls-server
tls-version-min 1.2
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
client-config-dir /etc/openvpn/ccd
status /var/log/openvpn/status.log
verb 3
log-append /var/log/openvpn/openvpn.log


Is there anything that could be improved?

Many Thanks,

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 10:23 am

TLS-Crypt-V2 should have zero impact on speed.

One thing that does have some impact is --auth SHA256
The default is SHA1 for a good reason and that is all that is required.
Fear of using SHA1 here is irrational.

However, that can only make a small difference and not the huge difference you are seeing.

For your server you could try the wintun driver.

But you don't give any details about your client ...

houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

Re: OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 12:10 pm

Hey buddy,

Sure, here is the client config:

client

auth-user-pass pass.txt
client
proto udp
explicit-exit-notify 3
remote xx.xx.xx.xx xxxx
dev tun
resolv-retry infinite
nobind
persist-key
persist-tun
remote-cert-tls server
verify-x509-name server_7RmgUuwxS0MOJqci name
auth SHA256
auth-nocache
cipher AES-128-GCM
tls-client
tls-version-min 1.2
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
ignore-unknown-option block-outside-dns
setenv opt block-outside-dns # Prevent Windows 10 DNS leak
verb 3
<ca>
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</ca>
<cert>
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
...
Signature Algorithm: ecdsa-with-SHA256
Issuer: CN=cn_pq0OCSJFFDgFzak6
Validity
Not Before: Jun 20 07:32:01 2021 GMT
Not After : Sep 23 07:32:01 2023 GMT
Subject: CN=client1
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
...
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Subject Key Identifier:
...
X509v3 Authority Key Identifier:
...
DirName:/CN=cn_pq0OCSJFFDgFzak6
serial:...

X509v3 Extended Key Usage:
TLS Web Client Authentication
X509v3 Key Usage:
Digital Signature
Signature Algorithm: ecdsa-with-SHA256
...
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgy7XWOjiOFgn1PAvd
XxfIvggtznx1ckQve+hQG6cllg6hRANCAARGlLBjirGj+1H2SzVXkDEmvDHyHwyu
O1Ue8USEFFgx3k42f7y3UZQK5NSUpIBgxTKq57sCiNpJcDx+rXS5SVjj
-----END PRIVATE KEY-----
</key>
<tls-crypt-v2>
-----BEGIN OpenVPN tls-crypt-v2 client key-----
...
-----END OpenVPN tls-crypt-v2 client key-----
</tls-crypt-v2>


Btw the Server is running on Debian 10.9 and Client is iPhone. I believe Wintun driver is only meant for Windows, if I'm not mistaken.

I found an article https://community.openvpn.net/openvpn/w ... wEGrEGtPow, which I'm trying to follow to see if it helps.

If you have any other idea, please let me know.

Thanks

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 2:08 pm

The Gigabit network article is useful but the best test you can do is run the client on the same LAN as the server (or as close as physically possible).

300000
OpenVPN Expert
Posts: 685
Joined: Tue May 01, 2012 9:30 pm

Re: OpenVPN server seems strangely slow

Post by 300000 » Sun Jun 20, 2021 3:01 pm

I run openvpn in tvbox with rk3328 cpu and the speed very fast. Download file 4.5 gb just under 5 minutes.

It is not cpu or openvpn but it is config make the most .
Image

Download file show about 30 MB so upload speed is over 230 Mb . Just check all again .

Image

Here is my server config and that yse the most up to date as show.

server 10.10.90.0 255.255.255.0
mute-replay-warnings
port 7800
tls-version-min "1.2" version
proto tcp4
txqueuelen 4500
tls-cipher TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384
tls-ciphersuites TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256
tls-cipher TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256:TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256:TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256:TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256
script-security 4
tls-timeout 90
hand-window 900
dev tun2
;dev-node "openvpn"
;ip-win32 manual
;route 10.10.90.0 255.255.255.0 10.10.90.1
push "redirect-gateway autolocal def1 "
push "ping 190"
socket-flags TCP_NODELAY
mute-replay-warnings
push "route-method exe"
push "socket-flags TCP_NODELAY"
remote-cert-eku "TLS Web Client Authentication"
remote-cert-tls client
topology subnet
mode server
hash-size 137552 155372
bcast-buffers 15379870
tcp-queue-limit 10000
tls-server
tun-mtu 48000
mssfix 0
fragment 0
sndbuf 3932106
rcvbuf 3932106
push "sndbuf 3932106"
push "rcvbuf 3932106"
push "route-method exe"
;push "route 192.168.20.0 255.255.255.0 vpn_gateway "
push "route 10.10.90.0 255.255.255.0 "
push "route 192.168.200.0 255.255.255.0"
push "route 192.168.5.0 255.255.255.0 vpn_gateway "
push "route 192.168.1.0 255.255.255.0 vpn_gateway "
push "route 192.168.170.0 255.255.255.0"
push "dhcp-option WINS 192.168.200.1"
push "dhcp-option DNS 192.168.200.1 "
push "dhcp-option NBT 2 "
push "dhcp-option DOMAIN-SEARCH xxx.xx.com"
push "dhcp-option DOMAIN xxx .xx.com"
push "persist-key "
push " persist-tun "
duplicate-cn
client-to-client
keepalive 90 190
persist-key
persist-tun
verb 1
mute 2
ecdh-curve secp521r1
status /var/log/openvpn/openvpn-status.log
log-append /var/log/openvpn.log
;show-engines
;show-curves
;show-tls
;show cipher
;mlock
data-ciphers-fallback AES-256-CBC
auth SHA512
dh none
<ca>
-----BEGIN CERTIFICATE-----
4zEucnlYK7CFpxQ/dosaoMhs39s=
-----END CERTIFICATE-----
</ca>
<cert>
-----BEGIN CERTIFICATE-----
G

-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN EC PRIVATE KEY-----
HevN
-----END EC PRIVATE KEY-----
</key>
<tls-crypt-v2>
-----BEGIN OpenVPN tls-crypt-v2 server key-----
gj
-----END OpenVPN tls-crypt-v2 server key-----
</tls-crypt-v2>
Last edited by 300000 on Sun Jun 20, 2021 4:54 pm, edited 2 times in total.

houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

Re: OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 3:16 pm

Hi,

Can you please share your server and client config with me?

How would I otherwise know how you have achieved this? I have shared mine above.
Many Thanks,

houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

Re: OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 3:36 pm

Thank you for sharing your server config.

It's a bit cluttered, I wonder how you maintain this. :-D
Multiple values are defined twice. It means the bottom one is overwriting the previous one. It's a bit hard to read.

To my surprise your server runs on TCP instead of UDP and UDP is meant to be faster.

The only thing I could extract from your settings is this:

server

mute-replay-warnings
txqueuelen 4500
tun-mtu 48000
mssfix 0
fragment 0


And this made the speed worse (12.0 Mbps).
It feels like OpenVPN server works better on Windows than on Linux. That's mind blowing. What am I doing wrong...

houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

Re: OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 4:08 pm

I have some news to share.

The same server reaches 150 Mbps on my Mac. (My fiber connection can't go faster than this) This means this is client CPU related.

An iPhone X is four years old and has a weaker CPU compared to a 2020 MacMini.

@TinCanTech

I think the data is encrypted twice, once by tls-crypt and once by the TLS session. It is safe to assume that the double encryption adds additional overhead to CPU on end-user devices like mobile phones because everything has to be decrypted twice. This would explain why the same server has a lower speed for iPhoneX but higher speed for MacMini with a faster CPU. Don't you agree?

User avatar
Pippin
Forum Team
Posts: 1201
Joined: Wed Jul 01, 2015 8:03 am
Location: irc://irc.libera.chat:6697/openvpn

Re: OpenVPN server seems strangely slow

Post by Pippin » Sun Jun 20, 2021 4:30 pm

No double encryption.
There are two channels, control channel and data channel.
https://build.openvpn.net/doxygen/

The manual describes which --<encryption directive> is used in which channel.
Just read about --tls-auth, --tls-crypt, --tls-crypt-v2, --auth and --cipher.
I gloomily came to the ironic conclusion that if you take a highly intelligent person and give them the best possible, elite education, then you will most likely wind up with an academic who is completely impervious to reality.
Halton Arp

300000
OpenVPN Expert
Posts: 685
Joined: Tue May 01, 2012 9:30 pm

Re: OpenVPN server seems strangely slow

Post by 300000 » Sun Jun 20, 2021 4:37 pm

houmie75 wrote:
Sun Jun 20, 2021 3:36 pm
Thank you for sharing your server config.

It's a bit cluttered, I wonder how you maintain this. :-D
Multiple values are defined twice. It means the bottom one is overwriting the previous one. It's a bit hard to read.

To my surprise your server runs on TCP instead of UDP and UDP is meant to be faster.

The only thing I could extract from your settings is this:

server

mute-replay-warnings
txqueuelen 4500
tun-mtu 48000
mssfix 0
fragment 0





And this made the speed worse (12.0 Mbps).
It feels like OpenVPN server works better on Windows than on Linux. That's mind blowing. What am I doing wrong...





Yes that is the best .

txqueuelen 4500. Default is 100 but make it 1000 or 4000 make it faster.
tun-mtu 48000. is the best for gigabye network .

mssfix 0
fragment 0

sndbuf 3932106
rcvbuf 3932106
push "sndbuf 3932106"
push "rcvbuf 3932106"

This is make is more stable and keep connection is good as it is.

Ping time is important .long ping time is slow . Short is best and tcp is the more suitable for all client on mobile network it is useable and I can download file over 4g network about 7 MB. That is over 50Mb . This is all it can give out and there is no complaints about speed anymore.

Please note I install it on armbian running on rockchip rk3328 very chip TV box and use only 3w power. It is not cpu make it slow but all other thing make it slow.
Last edited by 300000 on Sun Jun 20, 2021 5:10 pm, edited 2 times in total.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 5:03 pm

houmie75 wrote:
Sun Jun 20, 2021 4:08 pm
I think the data is encrypted twice, once by tls-crypt and once by the TLS session
Nope.
Pippin wrote:
Sun Jun 20, 2021 4:30 pm
No double encryption.
There are two channels, control channel and data channel.
Exactly.

TLS is only used on the control channel and accounts for an absolutely miniscule amount of data.

The data channel is encrypted by --data-cipher alone. I heard CHACHA20-POLY1305 is good for old mobiles ..
(--auth SHA1 in important for this reason - And many ciphers now do their own hashing built in)

User avatar
Pippin
Forum Team
Posts: 1201
Joined: Wed Jul 01, 2015 8:03 am
Location: irc://irc.libera.chat:6697/openvpn

Re: OpenVPN server seems strangely slow

Post by Pippin » Sun Jun 20, 2021 5:10 pm

Ah yes, also the new --data-cipher.
I gloomily came to the ironic conclusion that if you take a highly intelligent person and give them the best possible, elite education, then you will most likely wind up with an academic who is completely impervious to reality.
Halton Arp

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 5:14 pm

Pippin wrote:
Sun Jun 20, 2021 5:10 pm
Ah yes, also the new --data-cipher.
So good they named it twice 8-)

houmie75
OpenVPN Power User
Posts: 72
Joined: Wed Jul 22, 2020 7:46 pm

Re: OpenVPN server seems strangely slow

Post by houmie75 » Sun Jun 20, 2021 5:53 pm

I have

Code: Select all

tls-cipher TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256
in my server config, but not data-cipher.

Is that something introduced in OpenVPN 2.5? I can't find it in the 2.4 reference manual either.
TinCanTech wrote:
Sun Jun 20, 2021 5:03 pm

The data channel is encrypted by --data-cipher alone. I heard CHACHA20-POLY1305 is good for old mobiles ..
(--auth SHA1 in important for this reason - And many ciphers now do their own hashing built in)

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 6:40 pm

--data-ciphers is 2.5

User avatar
Pippin
Forum Team
Posts: 1201
Joined: Wed Jul 01, 2015 8:03 am
Location: irc://irc.libera.chat:6697/openvpn

Re: OpenVPN server seems strangely slow

Post by Pippin » Sun Jun 20, 2021 6:56 pm

Oh 2.4...
--cipher is deprecated in 2.5 and replaced by --data-cipher.

I don't know about iOS versions...

TinCanTech already wrote
One thing that does have some impact is --auth SHA256
The default is SHA1 for a good reason and that is all that is required.
Fear of using SHA1 here is irrational.
Although SHA256 is recommended (modernization and to avoid user questions) SHA1 is perfectly fine and safe in OpenVPN.

Please see --tls-cipher in manual and
The manual describes which --<encryption directive> is used in which channel.
Just read about --tls-auth, --tls-crypt, --tls-crypt-v2, --auth and --cipher.
If necessary multiple times, the penny will fall eventually :)

W.r.t. speed you can try and play with --sndbuf and --rcvbuf, it sometimes helps.

Stay away from messing with MTU though.
Generally you only turn those knobs if having connection problems and generally that would only be the --mssfix directive.
I gloomily came to the ironic conclusion that if you take a highly intelligent person and give them the best possible, elite education, then you will most likely wind up with an academic who is completely impervious to reality.
Halton Arp

300000
OpenVPN Expert
Posts: 685
Joined: Tue May 01, 2012 9:30 pm

Re: OpenVPN server seems strangely slow

Post by 300000 » Sun Jun 20, 2021 10:00 pm

please note I am not using Easy-RSA to create certificate so it is not limited to 10 years. My certificate authority
sign with new EEC key as picture. it will valid until year 4020. so it have life time 2000 years.
Image

Maybe with new EEC key for certificate authority it run faster.

Image

On the openvpn server create with new certificate not 10 years but 1000 years lifetime.

Image

The same using new EEC key with 512 bits for server.

Image

Openvpn client is the same as server

Image

EEC is shorter but the same strong and people like it. using old Signature algorithm slow it down a lot. Now is the time need to update new one.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 10:37 pm

300000 wrote:
Sun Jun 20, 2021 10:00 pm
My certificate authority
sign with new EEC key as picture. it will valid until year 4020. so it have life time 2000 years
Does not effect speed .. all that does is give your adversaries longer to hack you.

300000
OpenVPN Expert
Posts: 685
Joined: Tue May 01, 2012 9:30 pm

Re: OpenVPN server seems strangely slow

Post by 300000 » Sun Jun 20, 2021 10:43 pm

TinCanTech wrote:
Sun Jun 20, 2021 10:37 pm
300000 wrote:
Sun Jun 20, 2021 10:00 pm
My certificate authority
sign with new EEC key as picture. it will valid until year 4020. so it have life time 2000 years
Does not effect speed .. all that does is give your adversaries longer to hack you.
People is talking a lot on EEC and speed so I dont know but EEC is smaller than old one . how strong it is only let time to tell but it seem run fast .

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: OpenVPN server seems strangely slow

Post by TinCanTech » Sun Jun 20, 2021 11:03 pm

300000 wrote:
Sun Jun 20, 2021 10:43 pm
People is talking a lot on EEC and speed
EEC .. is that an elephantine-eaglesque-eggplantish-estrogenic-empathic-excited-esoteric-endomorphic-epileptic-empathic-embassysmic-embolismic-embalmed-emblazened-endocrinic-extreme-explicit-enshrined-ectoplasmic-engorged-endemic-endoscopic-endangered-endorsed-ezmoney elliptic curve ?

Post Reply