Issue with Linux double interface routing

hey1

Hi,

I've been trying to configure the OpenVPN server in a Debian system and I have some issues making it work. The server machine has two network interfaces: enp2s0 and enp3s0.

enp2s0: LAN network where the clients are connecting from (192.168.3.0).
enp3s0: private subnet. Consists of a single PC direct linked to the VPN machine (169.254.1.0).
tun0: the VPN interface (10.8.0.0).

IPs:
VPN machine: 192.168.3.222 and 169.254.1.5
VPN client: 192.168.3.22
Direct linked PC: 169.254.1.10

The aim of this VPN is to let the clients from 192.168.3.0 connect to 169.254.1.0. The problem is that I'm able to ping the private interface's IP address of the VPN machine from the client, but I can't ping the machine 169.254.1.10. I'm not able to figure it out where the issue stands. Probably, routing or the enp3s0's gateway? Here you can find the basic information:

server.conf file:

local 192.168.3.222
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
topology subnet
server 10.8.0.0 255.255.255.0
push "redirect-gateway def1 bypass-dhcp"
push "route 169.254.1.0 255.255.255.0"
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 10.8.0.1"
keepalive 10 120
cipher AES-256-CBC
user nobody
group nogroup
persist-key
persist-tun
verb 3
crl-verify crl.pem
explicit-exit-notify
log-append /var/log/openvpn.log

$ route -n:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.8.0.0        0.0.0.0         255.255.255.0   U     0      0        0 tun0
169.254.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp3s0
192.168.3.0     0.0.0.0         255.255.255.0   U     0      0        0 enp2s0

netplan yaml file:

network:
  ethernets:
    enp2s0:
      dhcp4: no
      addresses: [192.168.3.222/24]
      gateway4: 192.168.1.1
    enp3s0:
      dhcp4: no
      addresses: [169.254.1.5/24]
      gateway4: 169.254.1.5
  version: 2

You could probably find some inconsistencies in the interfaces configuration (desperate tries).

Please, if you need any additional information, let me know.

Thank you so much for the time!

hey1

I forgot to mention that IP forwarding is enabled and that I'm able to ping from the VPN server to the 169.254.1.10 machine.

hey1

I should point out that the mask of the enp2s0 interface is 255.255.252.0. That's why I have that gateway. The original yaml file is:

network:
  ethernets:
    enp2s0:
      dhcp4: no
      addresses: [192.168.3.222/22]
      gateway4: 192.168.1.1
    enp3s0:
      dhcp4: no
      addresses: [169.254.1.5/24]
  version: 2

Non of the posts or the articles worked to me. Any ideas? I'm now testing the Direct linked PC with another Linux system. I thought that Windows's firewall could be the problem. Nop. I'm still in the same situation.

The worst is that I previously managed to make the same VPN server work. I just don't know what am I missing.

Please, if anyone has any idea, please contact me. If you think there is any inconsistency in the configuration, let me know. I'll be glad to explain the reasons.
Thanks!

TinCanTech

You probably need --multihome in your server config.

hey1

Hello. Thank you for the response.

I updated the server conf file:

local 192.168.3.222
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
topology subnet
server 10.8.0.0 255.255.255.0
multihome
push "redirect-gateway def1 bypass-dhcp"
push "route 169.254.1.0 255.255.255.0"
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 10.8.0.1"
keepalive 10 120
cipher AES-256-CBC
user nobody
group nogroup
persist-key
persist-tun
verb 3
crl-verify crl.pem
explicit-exit-notify
log-append /var/log/openvpn.log

Still nothing. Maybe the iptables can give some needed info:

$iptables -S

-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -p udp -m udp --dport 1194 -j ACCEPT
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.8.0.0/24 -j ACCEPT

If direct linked, I'm not sure if the gateway of the interface enp3s0 should stay empty. I'm sure that when I set it up the last time, I didn't use the multihome option. I think I played with the gateway redirections/routing, but I can't find the key.

Thank you!

TinCanTech

Use tcpdump to see where the packets go.

hey1

Good morning,

I already did that. If I ping from the client to the server, the packets reach the server but not the direct linked PC. The server does not send the reply.

tcpdump ping request from the VPN client to the VPN server.
$tcpdump -i tun0:

07:47:42.980176 IP 10.8.0.2 > 169.254.1.10: ICMP echo request, id 1, seq 6793, length 40

However, if I ping the server's 169.254.1.5 address, the client receives the ping reply, but I'm not able to see specific packets using tcpdump in any of the three interfaces on the server. I only see this type of generic messages:

09:32:44.179580 IP 10.8.0.2 > vpnserver: ICMP echo request, id 1, seq 9393, length 40
09:32:44.179601 IP vpnserver > 10.8.0.2: ICMP echo reply, id 1, seq 9393, length 40

A server side clean log:

OpenVPN 2.4.7 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 19 2021
library versions: OpenSSL 1.1.1f  31 Mar 2020, LZO 2.10
Diffie-Hellman initialized with 2048 bit key
Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
TUN/TAP device tun0 opened
TUN/TAP TX queue length set to 100
/sbin/ip link set dev tun0 up mtu 1500
/sbin/ip addr add dev tun0 10.8.0.1/24 broadcast 10.8.0.255
Could not determine IPv4/IPv6 protocol. Using AF_INET
Socket Buffers: R=[212992->212992] S=[212992->212992]
UDPv4 link local (bound): [AF_INET]192.168.3.222:1194
UDPv4 link remote: [AF_UNSPEC]
GID set to nogroup
UID set to nobody
MULTI: multi_init called, r=256 v=256
IFCONFIG POOL: base=10.8.0.2 size=252, ipv6=0
ifconfig_pool_read(), in='new,10.8.0.2', TODO: IPv6
succeeded -> ifconfig_pool_set()
IFCONFIG POOL LIST
new,10.8.0.2
Initialization Sequence Completed
192.168.3.22:59854 TLS: Initial packet from [AF_INET]192.168.3.22:59854 (via [AF_INET]192.168.3.222%enp2s0), sid=56eded65 a41139bc
192.168.3.22:59854 VERIFY OK: depth=1, CN=ChangeMe
192.168.3.22:59854 VERIFY OK: depth=0, CN=new
192.168.3.22:59854 peer info: IV_VER=3.git::d3f8b18b
192.168.3.22:59854 peer info: IV_PLAT=win
192.168.3.22:59854 peer info: IV_NCP=2
192.168.3.22:59854 peer info: IV_TCPNL=1
192.168.3.22:59854 peer info: IV_PROTO=30
192.168.3.22:59854 peer info: IV_CIPHERS=AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305:AES-256-CBC
192.168.3.22:59854 peer info: IV_AUTO_SESS=1
192.168.3.22:59854 peer info: IV_GUI_VER=OCWindows_3.3.4-2600
192.168.3.22:59854 peer info: IV_SSO=webauth,openurl,crtext
192.168.3.22:59854 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
192.168.3.22:59854 [new] Peer Connection Initiated with [AF_INET]192.168.3.22:59854 (via [AF_INET]192.168.3.222%enp2s0)
new/192.168.3.22:59854 MULTI_sva: pool returned IPv4=10.8.0.2, IPv6=(Not enabled)
new/192.168.3.22:59854 MULTI: Learn: 10.8.0.2 -> new/192.168.3.22:59854
new/192.168.3.22:59854 MULTI: primary virtual IP for new/192.168.3.22:59854: 10.8.0.2
new/192.168.3.22:59854 PUSH: Received control message: 'PUSH_REQUEST'
new/192.168.3.22:59854 SENT CONTROL [new]: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,route 169.254.1.0 255.255.255.0,dhcp-option DNS 10.8.0.1,route-gateway 10.8.0.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.8.0.2 255.255.255.0,peer-id 0,cipher AES-256-GCM' (status=1)
new/192.168.3.22:59854 Data Channel: using negotiated cipher 'AES-256-GCM'
new/192.168.3.22:59854 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
new/192.168.3.22:59854 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
new/192.168.3.22:59854 IP packet with unknown IP version=0 seen

Any ideas?

Thanks for the help

Edit:
Confirmed that the "vpnserver" of the "generic message" I mentioned is the 169.254.1.5 address. I confirmed it routing directly the 169.254.1.5 and 169.254.1.10 addresses in the server config file:

route 169.254.1.10 255.255.255.255
route 169.254.1.5 255.255.255.255

The routing table remains like this:

10.8.0.0        0.0.0.0         255.255.255.0   U     0      0        0 tun0
169.254.1.0     0.0.0.0         255.255.255.0   U     0      0        0 enp3s0
vpnserver       10.8.0.2        255.255.255.255 UGH   0      0        0 tun0
169.254.1.10    10.8.0.2        255.255.255.255 UGH   0      0        0 tun0
192.168.0.0     0.0.0.0         255.255.252.0   U     0      0        0 enp2s0

I know this is wrong. Just wanted to try.

hey1

Any clue? I cannot figure it out.

hey1

Found the solution after days of struggling.

Just needed to add a single iptables rule that masquerades the second interface. Following this article (https://community.openvpn.net/openvpn/wiki/BridgingAndRouting), I found out that for my particular case, I only needed this rule:

iptables -t nat -I POSTROUTING -o enp3s0 -s 10.8.0.0/24 -j MASQUERADE

No --multihome needed.

TinCanTech

Your server is multi-homed, you do need --multihome.