openvpn3 client with openvpn2.4 server - possible?

mangoo

I have the following setup:

- server: openvpn 2.4.7 on Ubuntu 20.04
- client: openvpn 2.5.1 on Ubuntu 21.10
- client: openvpn 2.5.5 community on Windows

All traffic is redirected through the server (IPv4+nat and IPv6+nat), everything works with community 2.5 clients.

Now, if I try to use a openvpn3 client (17~beta2+hirsute) - it connects to the server, there are no warnings, no errors in client nor server log - but there is no internet on the client. It is possible to ping the server or connect to it with SSH, it is possible to ping other hosts in the internet, when openvpn3 client is connected, but everything else just times out.
When I check with tcpdump on the server - for example, when trying to "curl -v google.com" on the PC with openvpn3 client - I can see that the server is getting the reply from the google server, but somehow, the packets never make it to the client. With community openvpn 2.5 client, everything just works.

Not sure if that's expected - because there are no warnings nor errors in the log, I'm not sure how to debug this.

Perhaps openvpn2 server and openvpn3 client don't work together?

For reference - server config:

port 1194
proto udp
dev tun
ca ca.crt
cert issued/server.crt
key private/server.key # This file should be kept secret
dh dh.pem
tls-crypt ta.key

topology subnet
server 10.121.0.0 255.255.255.0

ifconfig-pool-persist /var/log/openvpn/ipp.txt

#push "route 10.11.0.0 255.255.255.0"
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 1.1.1.1"

# ipv6
server-ipv6 fd42:feed:feed:feed::/64
push "redirect-gateway ipv6 bypass-dhcp"

keepalive 10 120

cipher AES-256-CBC
compress lz4
persist-key
persist-tun
status /var/log/openvpn/openvpn-status.log
log-append /var/log/openvpn/openvpn.log
verb 3
explicit-exit-notify 1
auth SHA512

# allow the client to connect multiple times (i.e. PC and mobile)
duplicate-cn

# need to add client name to ccd dir
client-config-dir /etc/openvpn/ccd
ccd-exclusive

And client config:

client
dev tun
proto udp
remote aaa.bbb.ccc.ddd 1194
resolv-retry infinite

nobind
persist-key
persist-tun
remote-cert-tls server

compress lz4

cipher AES-256-CBC
auth SHA512

auth-nocache

verb 3

<ca>
...
</ca>

<key>
...
</key>

<cert>
...
</cert>

<tls-crypt>
...
</tls-crypt>

TinCanTech

mangoo wrote if I try to use a openvpn3 client (17~beta2+hirsute) -

Please see: https://github.com/OpenVPN/openvpn3

mangoo

TinCanTech wrote
mangoo wrote if I try to use a openvpn3 client (17~beta2+hirsute) -
Please see: https://github.com/OpenVPN/openvpn3

I can read "OpenVPN 3 is a C++ class library that implements the functionality of an OpenVPN client, and is protocol-compatible with the OpenVPN 2.x branch".

Is there any specific paragraph you wanted me to read?

TinCanTech

If you simply want help diagnosing a connection issue then
please document as described here. Which means, your logs please.

mangoo

Here are the logs when openvpn3 client is connecting to openvpn2 server. Let me know if you see any issues there.

- openvpn 2.4 server side log:

83.11.19.18:51034 TLS: Initial packet from [AF_INET]83.11.19.18:51034, sid=907b4642 f4576962
83.11.19.18:51034 VERIFY OK: depth=1, CN=Easy-RSA CA
83.11.19.18:51034 VERIFY OK: depth=0, CN=tch
83.11.19.18:51034 peer info: IV_VER=3.git:HEAD:7765540e
83.11.19.18:51034 peer info: IV_PLAT=linux
83.11.19.18:51034 peer info: IV_NCP=2
83.11.19.18:51034 peer info: IV_TCPNL=1
83.11.19.18:51034 peer info: IV_PROTO=30
83.11.19.18:51034 peer info: IV_CIPHERS=AES-128-GCM:AES-192-GCM:AES-256-GCM:CHACHA20-POLY1305
83.11.19.18:51034 peer info: IV_LZO_STUB=1
83.11.19.18:51034 peer info: IV_COMP_STUB=1
83.11.19.18:51034 peer info: IV_COMP_STUBv2=1
83.11.19.18:51034 peer info: IV_AUTO_SESS=1
83.11.19.18:51034 peer info: IV_GUI_VER=OpenVPN_3/Linux_v17_beta/3.git:HEAD:7765540e_linux_x86_64_64-bit
83.11.19.18:51034 peer info: IV_SSO=openurl,webauth
83.11.19.18:51034 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 4096 bit RSA
83.11.19.18:51034 [tch] Peer Connection Initiated with [AF_INET]83.11.19.18:51034
tch/83.11.19.18:51034 OPTIONS IMPORT: reading client specific options from: /etc/openvpn/ccd/tch
tch/83.11.19.18:51034 MULTI_sva: pool returned IPv4=10.121.0.4, IPv6=fd42:feed:feed:feed::1002
tch/83.11.19.18:51034 MULTI: Learn: 10.121.0.4 -> tch/83.11.19.18:51034
tch/83.11.19.18:51034 MULTI: primary virtual IP for tch/83.11.19.18:51034: 10.121.0.4
tch/83.11.19.18:51034 MULTI: Learn: fd42:feed:feed:feed::1002 -> tch/83.11.19.18:51034
tch/83.11.19.18:51034 MULTI: primary virtual IPv6 for tch/83.11.19.18:51034: fd42:feed:feed:feed::1002
tch/83.11.19.18:51034 PUSH: Received control message: 'PUSH_REQUEST'
tch/83.11.19.18:51034 SENT CONTROL [tch]: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,dhcp-option DNS 8.8.8.8,dhcp-option DNS 1.1.1.1,redirect-gateway ipv6 bypass-dhcp,tun-ipv6,route-gateway 10.121.0.1,topology subnet,ping 10,ping-restart 120,ifconfig-ipv6 fd42:feed:feed:feed::1002/64 fd42:feed:feed:feed::1,ifconfig 10.121.0.4 255.255.255.0,peer-id 2,cipher AES-256-GCM' (status=1)
tch/83.11.19.18:51034 Data Channel: using negotiated cipher 'AES-256-GCM'
tch/83.11.19.18:51034 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
tch/83.11.19.18:51034 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key

- openvpn3 client side log:

Feb  5 11:49:41 dell dbus-daemon[1035]: [system] Activating service name='net.openvpn.v3.sessions' requested by ':1.7890' (uid=1000 pid=1081007 comm="openvpn3 session-start --config tch.ov") (using servicehelper)
Feb  5 11:49:41 dell net.openvpn.v3.sessions[1081011]: OpenVPN 3/Linux v17_beta (openvpn3-service-sessionmgr)
Feb  5 11:49:41 dell net.openvpn.v3.sessions[1081011]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:41 dell net.openvpn.v3.sessions[1081011]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:41 dell dbus-daemon[1035]: [system] Activating service name='net.openvpn.v3.log' requested by ':1.7891' (uid=140 pid=1081011 comm="/usr/lib/x86_64-linux-gnu/openvpn3-linux/openvpn3-") (using servicehelper)
Feb  5 11:49:41 dell net.openvpn.v3.log[1081015]: OpenVPN 3/Linux v17_beta (openvpn3-service-logger)
Feb  5 11:49:41 dell net.openvpn.v3.log[1081015]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:41 dell net.openvpn.v3.log[1081015]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:41 dell net.openvpn.v3.log[1081015]: Idle exit set to 10 minutes
Feb  5 11:49:41 dell dbus-daemon[1035]: [system] Successfully activated service 'net.openvpn.v3.log'
Feb  5 11:49:41 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:3157490219064562092}  [:1.7891/net.openvpn.v3.sessions]
Feb  5 11:49:41 dell dbus-daemon[1035]: [system] Successfully activated service 'net.openvpn.v3.sessions'
Feb  5 11:49:42 dell dbus-daemon[1035]: [system] Activating service name='net.openvpn.v3.configuration' requested by ':1.7890' (uid=1000 pid=1081007 comm="openvpn3 session-start --config tch.ov") (using servicehelper)
Feb  5 11:49:42 dell net.openvpn.v3.configuration[1081021]: OpenVPN 3/Linux v17_beta (openvpn3-service-configmgr)
Feb  5 11:49:42 dell net.openvpn.v3.configuration[1081021]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:42 dell net.openvpn.v3.configuration[1081021]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:42 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:6006032589890898422}  [:1.7893/net.openvpn.v3.configuration]
Feb  5 11:49:42 dell dbus-daemon[1035]: [system] Successfully activated service 'net.openvpn.v3.configuration'
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: {tag:6006032589890898422} Config Manager INFO: Parsed single-use configuration 'tch.ovpn', owner: tch
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: {tag:6006032589890898422} Config Manager INFO: Setting configuration override 'persist-tun' to 'true' by UID 1000
Feb  5 11:49:43 dell dbus-daemon[1035]: [system] Activating service name='net.openvpn.v3.backends' requested by ':1.7891' (uid=140 pid=1081011 comm="/usr/lib/x86_64-linux-gnu/openvpn3-linux/openvpn3-") (using servicehelper)
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081029]: OpenVPN 3/Linux v17_beta (openvpn3-service-backendstart)
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081029]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081029]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:2664736647481760910}  [:1.7894/net.openvpn.v3.backends]
Feb  5 11:49:43 dell dbus-daemon[1035]: [system] Successfully activated service 'net.openvpn.v3.backends'
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081033]: Re-initiated process from pid 1081033 to backend process pid 1081034
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081034]: OpenVPN 3/Linux v17_beta (openvpn3-service-client)
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081034]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:43 dell net.openvpn.v3.backends[1081034]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:1754057063817800157}  [:1.7895/net.openvpn.v3.backends]
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:7152556800024388631}  [:1.7895/net.openvpn.v3.sessions]
Feb  5 11:49:43 dell dbus-daemon[1035]: [system] Activating service name='net.openvpn.v3.netcfg' requested by ':1.7895' (uid=140 pid=1081034 comm="/usr/lib/x86_64-linux-gnu/openvpn3-linux/openvpn3-") (using servicehelper)
Feb  5 11:49:43 dell net.openvpn.v3.netcfg[1081039]: Loading configuration file: /var/lib/openvpn3/netcfg.json
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:16141793784013189646}  [:1.7896/net.openvpn.v3.netcfg]
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: Logger VERB2: Attached: {tag:8214603996853596697}  [:1.7896/net.openvpn.v3.netcfg.core]
Feb  5 11:49:43 dell net.openvpn.v3.netcfg[1081039]: OpenVPN 3/Linux v17_beta (openvpn3-service-netcfg)
Feb  5 11:49:43 dell net.openvpn.v3.netcfg[1081039]: OpenVPN core 3.git:HEAD:7765540e linux x86_64 64-bit
Feb  5 11:49:43 dell net.openvpn.v3.netcfg[1081039]: Copyright (C) 2012-2020 OpenVPN Inc. All rights reserved.
Feb  5 11:49:43 dell dbus-daemon[1035]: [system] Successfully activated service 'net.openvpn.v3.netcfg'
Feb  5 11:49:43 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration VERB1: Redirect method: host-route
Feb  5 11:49:44 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Cleaning up resources for PID 1081034.
Feb  5 11:49:44 dell net.openvpn.v3.log[1081015]: {tag:1754057063817800157} Client INFO: Starting connection
Feb  5 11:49:44 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Socket protect called for socket 8, remote: 'aaa.bbb.ccc.ddd', tun: '', ipv6: no
Feb  5 11:49:44 dell net.openvpn.v3.log[1081015]: {tag:1754057063817800157} Client INFO: Connecting
Feb  5 11:49:45 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Virtual device '177725f5ta0cdt4bb0t9e75t34360f79acce' registered on /net/openvpn/v3/netcfg/1081034_177725f5ta0cdt4bb0t9e75t34360f79acce (owner uid 140, owner pid 1081034)
Feb  5 11:49:45 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Adding IP Adress 10.121.0.4/24 gw 10.121.0.1 ipv6: no
Feb  5 11:49:45 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Adding IP Adress fd42:feed:feed:feed::1002/64 gw fd42:feed:feed:feed::1 ipv6: yes
Feb  5 11:49:45 dell net.openvpn.v3.log[1081015]: {tag:16141793784013189646} Network Configuration INFO: Setting remote IP address to 'aaa.bbb.ccc.ddd ipv6: no
Feb  5 11:49:45 dell net.openvpn.v3.log[1081015]: {tag:1754057063817800157} Client INFO: Connected: aaa.bbb.ccc.ddd:1194 (aaa.bbb.ccc.ddd) via /UDPv4 on tun/10.121.0.4/fd42:feed:feed:feed::1002 gw=[10.121.0.1/fd42:feed:feed:feed::1]

With verb 4, the server also logs plenty of:

tch/83.11.19.18:53945 MULTI: bad source address from client [192.168.1.19], packet dropped

Which I suppose it why openvpn3 client does not have connectivity.
However, with exactly same config, openvpn2 client works just fine.
192.168.1.19 is a wi-fi address of the client.

TinCanTech

Your client looks to be successfully connected.

I can only presume that you have not setup your server correctly.

mangoo

I've attached the server config at the beginning of the thread.

OpenVPN clients version 2 connect just fine and have internet access.

OpenVPN clients version 3 connect using the same config, but don't have internet access.

I'd appreciate if you could point me into the ideas on "you have not setup your server correctly".

TinCanTech

https://www.ircpimps.org/redirect.png

mangoo

Thanks for the hints, but it doesn't explain why the setup works with openvpn2 clients, but not with openvpn3 clients.

For now, I'm assuming that version 3 clients are just that - beta.

TinCanTech

Can the Version 3 client ping the server VPN IP, at least ?

openvpn_inc

Hi mangoo,

Yes, you're right, it does not explain the issue. Are you trying the openvpn3 client on the same machine as openvpn 2.x?

You're also right, it's beta. Not a dirty word. It means the basic functionality is done, but some quirks exist. You can help with testing.

My guess is that something is wrong with the client's route table. Show us from the client:

ip addr ; ip route

Looks like you are using --redirect-gateway. This post might help with that.

regards, rob0

mangoo

TinCanTech wrote Can the Version 3 client ping the server VPN IP, at least ?

It can ping the VPN server using both the public and the VPN IP. It can also establish TCP connections with it (i.e. SSH) using both public and VPN IP.

It can also ping hosts in the internet - however, it can't establish TCP connections with them.

mangoo

openvpn_inc wrote Hi mangoo,

Yes, you're right, it does not explain the issue. Are you trying the openvpn3 client on the same machine as openvpn 2.x?

You're also right, it's beta. Not a dirty word. It means the basic functionality is done, but some quirks exist. You can help with testing.

My guess is that something is wrong with the client's route table. Show us from the client:
ip addr ; ip route
Looks like you are using --redirect-gateway. This post might help with that.

regards, rob0

Routes are identical on openvpn2 and openvpn3 clients:

0.0.0.0/1 via 10.121.0.1 dev tun0 
default via 192.168.1.1 dev wlp1s0 proto dhcp metric 600 
10.121.0.0/24 dev tun0 proto kernel scope link src 10.121.0.4 
10.230.28.0/24 dev lxdbr0 proto kernel scope link src 10.230.28.1 
128.0.0.0/1 via 10.121.0.1 dev tun0 
135.181.180.231 via 192.168.1.1 dev wlp1s0 
169.254.0.0/16 dev lxdbr0 scope link metric 1000 
192.168.1.0/24 dev wlp1s0 proto kernel scope link src 192.168.1.19 metric 600

For "ip addr", the only difference is that openvpn3 clients adds a broadcast address.

openvpn2 client:

397: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none 
    inet 10.121.0.5/24 scope global tun0
       valid_lft forever preferred_lft forever
    inet6 fd42:feed:feed:feed::1003/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::b525:7bf1:71df:baaa/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

openvpn3 client:

407: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500
    link/none 
    inet 10.121.0.6/24 brd 10.121.0.255 scope global tun0
       valid_lft forever preferred_lft forever
    inet6 fd42:feed:feed:feed::1004/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::abd2:6ff2:9cdc:9076/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

mangoo

openvpn_inc wrote Yes, you're right, it does not explain the issue. Are you trying the openvpn3 client on the same machine as openvpn 2.x?

Yes, I'm trying openvpn 2.x on the same Linux machine as openvpn3 (and of course I'm not running them both at the same time).

I've also tried openvpn3 on a Windows machine - no internet. With openvpn 2.x on the same Windows machine, everything works fine.

openvpn_inc

Hi mangoo,

It's interesting that you are seeing packets from 192.168.1.19 routed through the VPN, as if the 0/1 and 128/1 routes somehow override the 192.168.1/24 route. I don't see why that would be happening. I would not think the addition of "brd 10.121.0.255" could affect packet route selection.

Are there any iptables or nft rules on this client?

iptables-save -c

That should work also for nft because of iptables-nft(8). This client is Ubuntu 21.10, correct? That would be nft, so just to be sure,

nft show ruleset

I've asked openvpn3-linux developers to look in on this thread; perhaps they will have other suggestions.

regards, rob0

dazo

Hi, I'm the OpenVPN 3 Linux project lead and developer ... Can you please boost the --verb/log-level to 6 when testing with OpenVPN 3 Linux? The easiest way:

   $ openvpn2 --config CONFIG_FILE --verb 6

I see you're using AES-256-CBC here. This is actually considered a legacy cipher these days. Please use AES-256-GCM instead. Using that also removes the need to use "auth SHA512" (SHA512 is also not advisable, it adds an additional 32 bytes overhead with no security gain over SHA256 - Using SHA512 eats your tunnel performance). IIRC, using AES-256-GCM, the authentication overhead which is built into AES-GCM is just 12 bytes per packet all in all while SHA512 spends 64 bytes per packet.

In addition, your configuration uses "compress lz4". Please don't use compression. Most of the tunneled traffic is already compressed and the additional compression doesn't add anything useful. And it makes you vulnerable to the VORACLE attack vector. More details here: https://community.openvpn.net/openvpn/wiki/VORACLE

dazo

TinCanTech wrote
mangoo wrote if I try to use a openvpn3 client (17~beta2+hirsute) -
Please see: https://github.com/OpenVPN/openvpn3

You probably meant https://github.com/OpenVPN/openvpn3-linux

ordex

Hi mangoo,

mangoo wrote
openvpn_inc wrote Yes, you're right, it does not explain the issue. Are you trying the openvpn3 client on the same machine as openvpn 2.x?
Yes, I'm trying openvpn 2.x on the same Linux machine as openvpn3 (and of course I'm not running them both at the same time).

I've also tried openvpn3 on a Windows machine - no internet. With openvpn 2.x on the same Windows machine, everything works fine.

I don't want to create confusion, but I would like to suggest another option: can you please add

tun-mtu 1440

to your client config?
The MTU handling is a bit delicate at the moment (we are working on it) and it might be that openvpn3, with DCO, is attempting to use a too large frame size. (OpenVPN2 has extra quirks to deal with that, so you may not be exposed to the issue there)

Please let us know how it goes.

mangoo

ordex wrote tun-mtu 1440

Nope, adding that option didn't help.

mangoo

dazo wrote

   $ openvpn2 --config CONFIG_FILE --verb 6

Did you mean openvpn3 client above? openvpn 2.x client works just fine with a 2.x server.

It seems to use different syntax:

openvpn3 session-start --config CONFIG_FILE

As openvpn3 doesn't accept "--verb 6" argument, I've added "verb 6" to the config file.

However, the amount of data logged to /var/log/syslog does not seem to change and is minimal.