I've build a rpi cluster of 3 rpi 4 model b, an unmanaged switch and an external ssd, with one head node and two compute nodes. The two compute nodes are running without sd-card and been managed to netboot of the head node. It was all working quite well until i connected the head node to my OpenVPN server with a client called "cluster.ovpn". Suddenly the compute nodes of the cluster doesn't have internet connection when the head node is connected to the VPN.
Ifconfig on the head node:
$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.50.1 netmask 255.255.255.0 broadcast 192.168.50.255
inet6 fe80::a00e:4521:9de3:2b98 prefixlen 64 scopeid 0x20<link>
ether d8:3a:dd:b1:d8:dd txqueuelen 1000 (Ethernet)
RX packets 48712 bytes 8281518 (7.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 58080 bytes 68178016 (65.0 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.125 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::2e0:4cff:fe68:506 prefixlen 64 scopeid 0x20<link>
ether 00:e0:4c:68:05:06 txqueuelen 1000 (Ethernet)
RX packets 40640 bytes 4175701 (3.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13038 bytes 1408657 (1.3 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 879 bytes 121594 (118.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 879 bytes 121594 (118.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 10.8.0.2 netmask 255.255.255.0 destination 10.8.0.2
inet6 fe80:
d6b7:1a2:67f2 prefixlen 64 scopeid 0x20<link>unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 500 (UNSPEC)
RX packets 587 bytes 725202 (708.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4834 bytes 336543 (328.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.124 netmask 255.255.255.0 broadcast 192.168.0.255
inet6 fe80::77f5
9910:4d08 prefixlen 64 scopeid 0x20<link>ether d8:3a:dd:b1:d8:de txqueuelen 1000 (Ethernet)
RX packets 32407 bytes 3205873 (3.0 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3256 bytes 677595 (661.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Ip routes on the head node:
$ ip route show
0.0.0.0/1 via 10.8.0.1 dev tun0
default via 10.8.0.1 dev tun0
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.2
54.152.136.27 via 192.168.0.1 dev eth1
128.0.0.0/1 via 10.8.0.1 dev tun0
169.254.0.0/16 dev eth0 scope link src 169.254.232.99 metric 202
192.168.0.0/24 dev eth1 proto dhcp scope link src 192.168.0.125 metric 203
192.168.0.0/24 dev wlan0 proto dhcp scope link src 192.168.0.124 metric 304
192.168.50.0/24 dev eth0 proto kernel scope link src 192.168.50.1
Routing table on the head node:
$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.8.0.1 128.0.0.0 UG 0 0 0 tun0
0.0.0.0 10.8.0.1 0.0.0.0 UG 0 0 0 tun0
10.8.0.0 0.0.0.0 255.255.255.0 U 0 0 0 tun0
54.152.136.27 192.168.0.1 255.255.255.255 UGH 0 0 0 eth1
128.0.0.0 10.8.0.1 128.0.0.0 UG 0 0 0 tun0
169.254.0.0 0.0.0.0 255.255.0.0 U 202 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 203 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 304 0 0 wlan0
192.168.50.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
Cluster.ovpn file:
##############################################
# Sample client-side OpenVPN 2.0 config file #
# for connecting to multi-client server. #
# #
# This configuration can be used by multiple #
# clients, however each client should have #
# its own cert and key files. #
# #
# On Windows, you might want to rename this #
# file so it has a .ovpn extension #
##############################################
# Specify that we are a client and that we
# will be pulling certain config file directives
# from the server.
client
# Use the same setting as you are using on
# the server.
# On most systems, the VPN will not function
# unless you partially or fully disable
# the firewall for the TUN/TAP interface.
;dev tap
dev tun
# Windows needs the TAP-Win32 adapter name
# from the Network Connections panel
# if you have more than one. On XP SP2,
# you may need to disable the firewall
# for the TAP adapter.
;dev-node MyTap
# Are we connecting to a TCP or
# UDP server? Use the same setting as
# on the server.
;proto tcp
proto udp
# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
remote ec2-54-152-136-27.compute-1.amazonaws.com 1194
;remote my-server-2 1194
# Choose a random host from the remote
# list for load-balancing. Otherwise
# try hosts in the order specified.
;remote-random
# Keep trying indefinitely to resolve the
# host name of the OpenVPN server. Very useful
# on machines which are not permanently connected
# to the internet such as laptops.
resolv-retry infinite
# Most clients don't need to bind to
# a specific local port number.
nobind
# Downgrade privileges after initialization (non-Windows only)
user nobody
group nobody
# Try to preserve some state across restarts.
persist-key
persist-tun
# If you are connecting through an
# HTTP proxy to reach the actual OpenVPN
# server, put the proxy server/IP and
# port number here. See the man page
# if your proxy server requires
# authentication.
;http-proxy-retry # retry on connection failures
;http-proxy [proxy server] [proxy port #]
# Wireless networks often produce a lot
# of duplicate packets. Set this flag
# to silence duplicate packet warnings.
;mute-replay-warnings
# SSL/TLS parms.
# See the server config file for more
# description. It's best to use
# a separate .crt/.key file pair
# for each client. A single ca
# file can be used for all clients.
;ca ca.crt
;cert client.crt
;key client.key
# Verify server certificate by checking that the
# certificate has the correct key usage set.
# This is an important precaution to protect against
# a potential attack discussed here:
# http://openvpn.net/howto.html#mitm
#
# To use this feature, you will need to generate
# your server certificates with the keyUsage set to
# digitalSignature, keyEncipherment
# and the extendedKeyUsage to
# serverAuth
# EasyRSA can do this for you.
remote-cert-tls server
# If a tls-auth key is used on the server
# then every client must also have the key.
;tls-auth ta.key 1
# Select a cryptographic cipher.
# If the cipher option is used on the server
# then you must also specify it here.
# Note that v2.4 client/server will automatically
# negotiate AES-256-GCM in TLS mode.
# See also the data-ciphers option in the manpage
;cipher AES-256-CBC
cipher AES-256-GCM
auth SHA256
key-direction 1
# Enable compression on the VPN link.
# Don't enable this unless it is also
# enabled in the server config file.
#comp-lzo
# Set log file verbosity.
verb 3
# Silence repeating messages
;mute 20
I've tried a lot of things before writing this post such as:
1. Check Routing Configuration on Head Node: Ensure that the routing configuration on the head node is correctly set up to forward traffic from the internal cluster network (192.168.50.0/24) to the VPN tunnel (tun0).
2. Inspect Firewall Rules on Head Node: Double-check the firewall rules on the head node (iptables rules) to ensure that they're not blocking outgoing traffic from the internal cluster network.
3. Check OpenVPN Server Configuration: Verify the configuration of the OpenVPN server to ensure that it's correctly configured to handle traffic from the client (head node) and that it's not blocking internet access for connected clients.
4. Configure NAT on the head node:
sudo iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
sudo iptables -A FORWARD -i eth1 -o tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i tun0 -o eth1 -j ACCEPT
And many more things, maybe somethings that made it worse...
As a final attempt on trying to make this work, hopefully one of the bright minds in here has a solution or the guidance to lead me in the right direction.
If there is anything else i can provide you with to enlighten you, please say so and i will post it below!
Thanks in advance!