Hi, I have a home network consisting of 3 servers and some masqueraded devices.
Servers are connected using OVPN. All works well except for one network, in which I can only reach devices with IP from 10.0.0.1 to 10.0.0.10.
For example when I ping from a device in 10.123.45.0/24 network a device in 10.0.0.0/24 network with IP < 10.0.0.10 for example 10.0.0.2 I get:
$ ping -c 3 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=62 time=53.2 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=62 time=53.7 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=62 time=53.3 ms
--- 10.0.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 53.253/53.464/53.782/0.296 ms
from the same host, when I ping something with IP > 10.0.0.10 I get:
$ ping -c 3 10.0.0.11
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.
--- 10.0.0.11 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms
when I ping
from within 10.0.0.0/24 network it all works well:
$ ping 10.0.0.11 -c 3
PING 10.0.0.11 (10.0.0.11) 56(84) bytes of data.
64 bytes from 10.0.0.11: icmp_seq=1 ttl=64 time=0.297 ms
64 bytes from 10.0.0.11: icmp_seq=2 ttl=64 time=0.281 ms
64 bytes from 10.0.0.11: icmp_seq=3 ttl=64 time=0.250 ms
--- 10.0.0.11 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.250/0.276/0.297/0.019 ms
It all works well if I try from the other side 10.0.0.0/24 -> 10.123.45.0/24
Here pinging my mobile phone from outside server
$ ping -c 3 10.123.45.132
PING 10.123.45.132 (10.123.45.132) 56(84) bytes of data.
64 bytes from 10.123.45.132: icmp_seq=1 ttl=63 time=367 ms
64 bytes from 10.123.45.132: icmp_seq=2 ttl=63 time=264 ms
64 bytes from 10.123.45.132: icmp_seq=3 ttl=63 time=308 ms
--- 10.123.45.132 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 264.573/313.563/367.595/42.211 ms
crude schematics:
VPS - server configs:
port PORT_NUMBER
proto tcp
dev tun
ca /PATH_TO_KEYS/ca.crt
cert /PATH_TO_KEYS/server.crt
key /PATH_TO_KEYS/server.key
dh /PATH_TO_KEYS/pem.pem
server 10.123.50.0 255.255.255.0
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
persist-key
persist-tun
status log/openvpn-status.log
log log/openvpn.log
log-append log/openvpn.log
verb 3
client-config-dir networks
route 10.123.45.0 255.255.255.0
route 10.0.0.0 255.255.255.0
client-to-client
push "route 10.123.45.0 255.255.255.0"
push "route 10.123.50.0 255.255.255.0"
push "route 10.0.0.0 255.255.255.0"
push "dhcp-option DNS 127.0.0.1"
push "dhcp-option DNS 8.8.8.8"
VPS - server configs - site W client config:
iroute 10.123.45.0 255.255.255.0
VPS - server configs - site O client config:
iroute 10.0.0.0 255.255.255.0
VPS - routing:
$ ip route list
default via _PUBLIC_IP_ dev eth0
10.0.0.0/24 via 10.123.50.2 dev tun0
10.123.45.0/24 via 10.123.50.2 dev tun0
10.123.50.0/24 via 10.123.50.2 dev tun0
10.123.50.2 dev tun0 proto kernel scope link src 10.123.50.1
_PUBLIC_NETWORK_ dev eth0 proto kernel scope link src _PUBLIC_IP_2_
VPN - tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.123.50.1 P-t-P:10.123.50.2 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:20118 errors:0 dropped:0 overruns:0 frame:0
TX packets:19920 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:1296611 (1.2 MiB) TX bytes:3999732 (3.8 MiB)
site W - routing:
$ ip route list
default via _PUBLIC_IP_ dev enp7s1 metric 2
10.0.0.0/24 via 10.123.50.5 dev tun0
10.123.45.0/24 dev br0 proto kernel scope link src 10.123.45.1
10.123.50.0/24 via 10.123.50.5 dev tun0
10.123.50.5 dev tun0 proto kernel scope link src 10.123.50.6
_PUBLIC_NETWORK_ dev enp7s1 proto kernel scope link src _PUBLIC_IP_2_ metric 2
127.0.0.0/8 dev lo scope host
site W - tun0
tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500
inet 10.123.50.6 netmask 255.255.255.255 destination 10.123.50.5
unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 100 (UNSPEC)
RX packets 666 bytes 120585 (117.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1117 bytes 82675 (80.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
site W - openvpn.conf
client
dev tun
port PORT_NUMBER
proto tcp
remote VPN_IP_ADDRESS PORT_NUMBER
nobind
ca /PATH_TO_KEYS/ca.crt
cert /PATH_TO_KEYS/site_w.crt
key /PATH_TO_KEYS/site_w.key
comp-lzo
persist-key
persist-tun
site O - openvpn.conf
client
dev tun
port PORT_NUMBER
proto tcp
remote VPN_IP_ADDRESS PORT_NUMBER
nobind
ca /PATH_TO_KEYS/ca.crt
cert /PATH_TO_KEYS/site_o.crt
key /PATH_TO_KEYS/site_o.key
comp-lzo
persist-key
persist-tun
site O - tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.123.50.10 P-t-P:10.123.50.9 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:19981 errors:0 dropped:0 overruns:0 frame:0
TX packets:19735 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:3952704 (3.7 MiB) TX bytes:1297245 (1.2 MiB)
site O - routing
$ ip route list
default via _PUBLIC_IP_ dev eth0
10.0.0.0/24 dev br0 proto kernel scope link src 10.0.0.1
10.123.45.0/24 via 10.123.50.9 dev tun0
10.123.50.0/24 via 10.123.50.9 dev tun0
10.123.50.9 dev tun0 proto kernel scope link src 10.123.50.10
169.254.0.0/16 dev eth0 scope link metric 1000
_PUBLIC_NETWORK_ dev eth0 proto kernel scope link src _PUBLIC_IP_2_
this is a tcpdump made on server in site O, while pinging masqueraded host from 10.123.45.0/24 network:
$ tcpdump -i tun0 src 10.0.0.11 or dst 10.0.0.11
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
21:56:48.119583 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 1, length 64
21:56:49.116886 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 2, length 64
21:56:50.118425 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 3, length 64
...
^C
15 packets captured
20 packets received by filter
4 packets dropped by kernel
same, different interface:
$ tcpdump -i br0 src 10.0.0.11 or dst 10.0.0.11
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:57:27.115113 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 40, length 64
21:57:28.115182 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 41, length 64
21:57:29.117355 IP 10.123.45.2 > 10.0.0.11: ICMP echo request, id 8048, seq 42, length 64
...
^C
13 packets captured
17 packets received by filter
3 packets dropped by kernel
the same server - site O pinging 10.0.0.10 from site W (for comparison)
$ sudo tcpdump -i br0 src 10.123.45.2 or dst 10.123.45.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:12:49.432797 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 11, length 64
22:12:49.432998 IP 10.0.0.10 > 10.123.45.2: ICMP echo reply, id 8156, seq 11, length 64
22:12:50.434650 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 12, length 64
22:12:50.434839 IP 10.0.0.10 > 10.123.45.2: ICMP echo reply, id 8156, seq 12, length 64
22:12:51.436567 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 13, length 64
22:12:51.436781 IP 10.0.0.10 > 10.123.45.2: ICMP echo reply, id 8156, seq 13, length 64
...
^C
20 packets captured
28 packets received by filter
6 packets dropped by kernel
same thing different interface
$ sudo tcpdump -i tun0 src 10.123.45.2 or dst 10.123.45.2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
22:14:29.562307 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 111, length 64
22:14:29.562576 IP 10.0.0.10 > 10.123.45.2: ICMP echo reply, id 8156, seq 111, length 64
22:14:30.565546 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 112, length 64
22:14:30.565856 IP 10.0.0.10 > 10.123.45.2: ICMP echo reply, id 8156, seq 112, length 64
22:14:31.621694 IP 10.123.45.2 > 10.0.0.10: ICMP echo request, id 8156, seq 113, length 64
...
^C
49 packets captured
61 packets received by filter
6 packets dropped by kernel
