I am trying to get a customer's tower site on VPN for the purposes of remote device command/control. It is a very remote site and cell is my only option there, so an Inseego FX3100 router was purchased from their cell provider. It is not possible for me to connect to this router from the outside world (in spite of promises to the contrary during the sale), but it is equipped with embedded OpenVPN, so I thought I would give you guys a try (over the decades, I have done PPTP and/or IPsec solutions for this, but always keen to add new tools to the bag).
For a quick deployment test, I grabbed a script online that automates key and config generation (yes, I looked it over first, I'm in a time crunch, not stupid), and sure enough, the tunnels are there and are working. For the moment, I have OpenVPN running at the tower site and my home domain, since that's the only domain I have full control over. The tunnel between works as designed, things at the tower site can access things on my home network. Now I need it to also work the other way around. From my home, I can reach the Inseego router's tunnel endpoint of 10.8.0.2, but nothing beyond that.
I have followed the guidance I can find for expanding client-side access, but so far am having no luck. I have tried multiple approaches but can't be certain the router is even getting all the changes pushed to it (and it's a 90 minute drive to the site). It's at the point where I'm basically throwing jell-o at the walls and hoping it sticks, I really need a good proof-read here, and some troubleshooting guidance.
The current topography is:
OpenVPN Server, home domain, 192.168.2.0/24, endpoint 10.8.0.1
OpenVPN router, remote site, 192.168.97.0/24, endpoint 10.8.0.2, designated "txsite"
My Win10 laptop, road warrior, endpoint 10.8.0.3, designated "trlaptop"
Once this part works, I will add:
OpenVPN client, customer site, 192.168.1.0/24, designated "studio"
Ultimately, I need all resources at the "txsite" endpoint to be addressable by all members of the VPN, the logical first step is getting it accessible from the server. For now, client-to-client is in effect, and all three 10.8.0.x addresses can ping each other. It's almost there, I just need that last hop to happen.
So here's the state of the current configs, I'm hoping there's a stupid and easy-to-spot mistake here:
Server Config
local 192.168.2.8
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
client-config-dir ccd
client-to-client
server 10.8.0.0 255.255.255.0
topology subnet
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
push "block-outside-dns"
push "route 192.168.1.0 255.255.255.0"
push "route 192.168.2.0 255.255.255.0"
push "route 192.168.97.0 255.255.255.0"
route 192.168.1.0 255.255.255.0
route 192.168.2.0 255.255.255.0
route 192.168.97.0 255.255.255.0
keepalive 10 120
user nobody
group nogroup
persist-key
persist-tun
verb 3
crl-verify crl.pem
explicit-exit-notify
Version information for the server
Code: Select all
# openvpn --version
OpenVPN 2.5.5 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Jul 14 2022
library versions: OpenSSL 3.0.2 15 Mar 2022, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2021 OpenVPN Inc <sales@openvpn.net>
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto_ofb_cfb=yes enable_debug=yes enable_def_auth=yes enable_dependency_tracking=no enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=needless enable_fragment=yes enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_maintainer_mode=no enable_management=yes enable_multihome=yes enable_option_checking=no enable_pam_dlopen=no enable_pedantic=no enable_pf=yes enable_pkcs11=yes enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=yes enable_werror=no enable_win32_dll=yes enable_x509_alt_username=yes with_aix_soname=aix with_crypto_library=openssl with_gnu_ld=yes with_mem_check=no with_sysroot=no
Code: Select all
# uname -a
Linux MYHOST 5.15.0-89-generic #99-Ubuntu SMP Mon Oct 30 20:42:41 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Code: Select all
# cat /etc/issue
Ubuntu 22.04.3 LTS \n \l
ccd txsite
iroute 192.168.97.0 255.255.255.0
iroute 0.0.0.0 0.0.0.0
(The 0.0.0.0 clause is the latest "maybe that's it", no luck)
txsite.ovpn
client
dev tun
proto udp
remote MYDOMAIN.net 1194
resolv-retry infinite
nobind
persist-key
persist-tun
remote-cert-tls server
auth SHA512
ignore-unknown-option block-outside-dns
verb 3
Since the question will come up, ip routing on the server:
Code: Select all
# ip route
default via 192.168.2.1 dev eth0
default via 192.168.2.1 dev eth0 proto static metric 100
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.1
192.168.2.0/24 dev eth0 proto kernel scope link src 192.168.2.8
192.168.97.0/24 via 10.8.0.2 dev tun0 src 10.8.0.1Code: Select all
# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward
-N ufw-after-logging-input
-N ufw-after-logging-output
-N ufw-after-output
-N ufw-before-forward
-N ufw-before-input
-N ufw-before-logging-forward
-N ufw-before-logging-input
-N ufw-before-logging-output
-N ufw-before-output
-N ufw-reject-forward
-N ufw-reject-input
-N ufw-reject-output
-N ufw-track-forward
-N ufw-track-input
-N ufw-track-output
-A INPUT -p udp -m udp --dport 1194 -j ACCEPT
-A INPUT -j ufw-before-logging-input
-A INPUT -j ufw-before-input
-A INPUT -j ufw-after-input
-A INPUT -j ufw-after-logging-input
-A INPUT -j ufw-reject-input
-A INPUT -j ufw-track-input
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 10.8.0.0/24 -j ACCEPT
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A FORWARD -j ufw-track-forward
-A FORWARD -s 192.168.2.0/24 -d 192.168.97.0/24 -i tun0 -j ACCEPT
-A FORWARD -s 192.168.2.0/24 -d 10.8.0.0/24 -i tun0 -j ACCEPT
-A OUTPUT -j ufw-before-logging-output
-A OUTPUT -j ufw-before-output
-A OUTPUT -j ufw-after-output
-A OUTPUT -j ufw-after-logging-output
-A OUTPUT -j ufw-reject-output
-A OUTPUT -j ufw-track-output
While we're at it, the log from the router, from the last time I restarted the tunnel at the remote site (in its current configuration, server IP address redacted with MYDOMAIN):
Code: Select all
Sat Dec 2 02:35:33 2023 WARNING: file 'openvpn_auth_configure' is group or others accessible
Sat Dec 2 02:35:33 2023 OpenVPN 2.4.9 arm-oe-linux-gnueabi [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD] built on Apr 15 2023
Sat Dec 2 02:35:33 2023 library versions: OpenSSL 1.1.1l 24 Aug 2021, LZO 2.10
Sat Dec 2 02:35:33 2023 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Sat Dec 2 02:35:33 2023 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sat Dec 2 02:35:33 2023 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sat Dec 2 02:35:33 2023 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sat Dec 2 02:35:33 2023 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sat Dec 2 02:35:33 2023 TCP/UDP: Preserving recently used remote address: [AF_INET]MYDOMAIN:1194
Sat Dec 2 02:35:33 2023 Socket Buffers: R=[1048576->1048576] S=[1048576->1048576]
Sat Dec 2 02:35:33 2023 UDP link local: (not bound)
Sat Dec 2 02:35:33 2023 UDP link remote: [AF_INET]MYDOMAIN:1194
Sat Dec 2 02:35:33 2023 TLS: Initial packet from [AF_INET]MYDOMAIN:1194, sid=b69700a5 36d8094d
Sat Dec 2 02:35:33 2023 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sat Dec 2 02:35:33 2023 VERIFY OK: depth=1, CN=Easy-RSA CA
Sat Dec 2 02:35:33 2023 VERIFY KU OK
Sat Dec 2 02:35:33 2023 Validating certificate extended key usage
Sat Dec 2 02:35:33 2023 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Sat Dec 2 02:35:33 2023 VERIFY EKU OK
Sat Dec 2 02:35:33 2023 VERIFY OK: depth=0, CN=server
Sat Dec 2 02:35:33 2023 WARNING: 'cipher' is present in local config but missing in remote config, local='cipher BF-CBC'
Sat Dec 2 02:35:33 2023 Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, 2048 bit RSA
Sat Dec 2 02:35:33 2023 [server] Peer Connection Initiated with [AF_INET]MYDOMAIN:1194
Sat Dec 2 02:35:34 2023 SENT CONTROL [server]: 'PUSH_REQUEST' (status=1)
Sat Dec 2 02:35:34 2023 PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1 bypass-dhcp,dhcp-option DNS 8.8.8.8,dhcp-option DNS 8.8.4.4,block-outside-dns,route-gateway 10.8.0.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.8.0.2 255.255.255.0,peer-id 0,cipher AES-256-GCM'
Sat Dec 2 02:35:34 2023 Unrecognized option or missing or extra parameter(s) in [PUSH-OPTIONS]:4: block-outside-dns (2.4.9)
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: timers and/or timeouts modified
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: --ifconfig/up options modified
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: route options modified
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: route-related options modified
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: peer-id set
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: adjusting link_mtu to 1624
Sat Dec 2 02:35:34 2023 OPTIONS IMPORT: data channel crypto options modified
Sat Dec 2 02:35:34 2023 Data Channel: using negotiated cipher 'AES-256-GCM'
Sat Dec 2 02:35:34 2023 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sat Dec 2 02:35:34 2023 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sat Dec 2 02:35:34 2023 ROUTE_GATEWAY ON_LINK IFACE=rmnet_data0 HWADDR=00:00:00:00:00:00
Sat Dec 2 02:35:34 2023 TUN/TAP device tun0 opened
Sat Dec 2 02:35:34 2023 TUN/TAP TX queue length set to 100
Sat Dec 2 02:35:34 2023 /sbin/ip link set dev tun0 up mtu 1500
Sat Dec 2 02:35:34 2023 /sbin/ip addr add dev tun0 10.8.0.2/24 broadcast 10.8.0.255
Sat Dec 2 02:35:34 2023 openvpn-up.sh tun0 1500 1552 10.8.0.2 255.255.255.0 init
Sat Dec 2 02:35:34 2023 /sbin/ip route add MYDOMAIN/32 dev rmnet_data0
Sat Dec 2 02:35:34 2023 /sbin/ip route add 0.0.0.0/1 via 10.8.0.1
Sat Dec 2 02:35:34 2023 /sbin/ip route add 128.0.0.0/1 via 10.8.0.1
Sat Dec 2 02:35:34 2023 Initialization Sequence Completed
For what it's worth, I can change this to a TAP approach if I have to (it would actually be much more straightforward), but would really like to get this working in TUN mode if possible.
Thanks in advance for whatever config proofreading and/or troubleshooting tips you can give me, I have beat on this for way too long and am running out of time.
--Thom