Msg not returning. Route problem

[thalesmaoa]

Hi there! I've read a lot of posts and troubleshooting regarding my problem. Unfortunately, I can't figure out where the problem is. I can really use some help.

I'm trying to reach the network of a client.

Server -> Client -> Device inside Client LAN

Expanding the scope of the VPN to include additional machines on either the client or server subnet

My server is a VPS from Hostinger.

View OriginalServer Config

#server.conf
local 185.28.23.108
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
topology subnet
push "redirect-gateway def1 bypass-dhcp"
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 31.220.105.10"
push "dhcp-option DNS 9.9.9.9"
push "block-outside-dns"
keepalive 10 120
cipher AES-256-CBC
user nobody
group nogroup
persist-key
persist-tun
verb 3
crl-verify crl.pem
explicit-exit-notify

server 10.254.254.0 255.255.255.0
client-to-client
client-config-dir /etc/openvpn/ccd

# Advertise subnet of VPS
push "route 185.28.23.0 255.255.255.0"

# Route for client-to-client
route 192.168.200.0 255.255.255.0
push "route 192.168.200.0 255.255.255.0"

View Originalccd

# ccd client
ifconfig-push 10.254.254.2 255.255.255.0
iroute 192.168.200.0 255.255.255.0

With that, from the VPS (185.28.23.108) I try to ping 192.168.200.1.

Using tcpdump I was able to see the package going through tun0 interface (VPS -> Client). From client to internal network (Client -> Second device).
However, the package doesn't return.

Code: Select all

# Dump at client side
sudo tcpdump -i utun3 icmp
01:26:42.925415 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 842, seq 612, length 64

Code: Select all

# Dump at client side
sudo tcpdump -i en0 icmp
01:36:59.296662 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 859, seq 1, length 64

My thought is that route is not making it back to the server. However, it is routed.

Code: Select all

185.28.23/24       10.254.254.1       UGSc            utun3       
185.28.23.108/32   192.168.200.1      UGSc              en0       

Any suggestion?

[300000]

If you like to access devices inside client lan you need perform ip forward and NAT at client openvpn . Your server and all route info but it stop at client as it need routing between virtual network and client lan network.

[thalesmaoa]

Thanks! I'm aware of that. I think I got it write, but probably I'm missing something.

Please, if possible, check my route table:

From server side:

Code: Select all

thales@vps:~$ ip r
default dev venet0 scope link 
10.254.254.0/24 dev tun0 proto kernel scope link src 10.254.254.1 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 
172.18.0.0/16 dev br-d28d8278c890 proto kernel scope link src 172.18.0.1 
185.28.23.0/24 dev venet0 proto kernel scope link src 185.28.23.109 
192.168.200.0/24 via 10.254.254.2 dev tun0

It seems correct. All 192.168.200.0/24 is being redirected to the client 10.254.254.2. This is why I see the package going out.

At the client side:

Code: Select all

user@user-virtual-machine:~$ ip r
default via 10.254.254.1 dev tun0 proto static metric 50 
default via 192.168.200.1 dev ens33 proto dhcp metric 20100 
10.254.254.0/24 dev tun0 proto kernel scope link src 10.254.254.2 metric 50 
169.254.0.0/16 dev ens33 scope link metric 1000 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
185.28.23.0/24 via 10.254.254.1 dev tun0 proto static metric 50 
185.28.23.109 via 192.168.200.1 dev ens33 proto static metric 100 
192.168.200.0/24 via 10.254.254.1 dev tun0 proto static metric 50 
192.168.200.0/24 dev ens33 proto kernel scope link src 192.168.200.87 metric 100 
192.168.200.1 dev ens33 proto static scope link metric 100

It also seems correct. When trying to reach my VPS 185.28.33.0/24, it uses the server IP 10.254.254.1.

Also, the traffic is being redirected:

Code: Select all

user@user-virtual-machine:~$ cat /proc/sys/net/ipv4/ip_forward
1

At last, NAT at client side:

Code: Select all

user@user-virtual-machine:~$ sudo iptables -t nat -A POSTROUTING -s 192.168.200.0/24 -d 10.254.254.0/24 -j MASQUERADE

Do you see any problem?

[thalesmaoa]

Just another quick update!

I've missed this part:

Similarly, if the client machine running OpenVPN is not also the gateway for the client LAN, then the gateway for the client LAN must have a route which directs all subnets which should be reachable through the VPN to the OpenVPN client machine.

Ok! I'm confused. I saw so many successful attempts over the internet. Do I really need to make the client the gateway of my LAN?

I have a gateway with ip 192.168.200.254. I'm using a client 192.168.200.78 to connect VPN.
I'm trying to ping 192.168.200.254 over the tun0 interface from VPS.

Code: Select all

thales@elementaryDesktopCasa:~$ sudo tcpdump -i tun0 'icmp[icmptype] = icmp-echo or icmp[icmptype] = icmp-echoreply'
[sudo] senha para thales:        
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 262144 bytes
17:18:14.053143 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 2710, seq 1, length 64
17:18:15.051877 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 2710, seq 2, length 64
17:18:16.051873 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 2710, seq 3, length 64
17:18:17.051864 IP 185.28.23.109 > 192.168.200.254: ICMP echo request, id 2710, seq 4, length 64

Traffic is been redirected over tun interface, but not over eno4s0 interface:

Code: Select all

thales@elementaryDesktopCasa:~$ sudo tcpdump -i enp4s0 'icmp[icmptype] = icmp-echo or icmp[icmptype] = icmp-echoreply'
[sudo] senha para thales:        
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on enp4s0, link-type EN10MB (Ethernet), capture size 262144 bytes

[thalesmaoa]

More updates!

I found that port forwarding was not working in the tested machine.
After changing machine, enabling port forward:

Code: Select all

sudo nano /etc/sysctl.conf

The icmp request is passing through and reaching the client. After that, I've MASQUERADE the outbound.

Come from tun0

Code: Select all

15:13:06.700261 IP 185.28.23.108 > 192.168.200.254: ICMP echo request, id 1059, seq 1, length 64

Pass to enp4s0

Code: Select all

15:11:50.846663 IP elementaryDesktopCasa > 192.168.200.254: ICMP echo request, id 1052, seq 258, length 64

Reaches 192.168.200.254 and return

Code: Select all

15:13:10.146514 IP 192.168.200.65 > 192.168.200.254: ICMP echo request, id 56026, seq 15361, length 64
15:13:10.146625 IP 192.168.200.254 > 192.168.200.65: ICMP echo reply, id 56026, seq 15361, length 64

Reaches VPN client

Code: Select all

15:13:06.701093 IP 192.168.200.254 > elementaryDesktopCasa: ICMP echo reply, id 1059, seq 1, length 64
15:13:06.701114 IP 192.168.200.254 > 185.28.23.109: ICMP echo reply, id 1059, seq 1, length 64

But server don't get the package.

EDIT:

One last update, I noticed that the problem is now from server side.
I'm using SNAT rule instead of MASQUERADE. It should work, but it doesn't.

What is the equivalence of

Code: Select all

ptables -t nat -D POSTROUTING -j MASQUERADE

using SNAT?

[300000]

just try two iptables with root to see if it work for you.




iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.254.254.0/24 -j SNAT --to 192.168.200.78

[thalesmaoa]

Ok, I will keep the updates to help other.

I managed to translate MASQUERADE to SNAT with your help

In the client

Code: Select all

sudo iptables -t nat -A POSTROUTING -s 185.28.23.108 -d 192.168.200.1/24 -o enp4s0 -j SNAT --to 192.168.200.78

However, it can't route back to 185.28.23.108.

SNAT in server

Code: Select all

sudo iptables -t nat -A POSTROUTING -s 10.254.254.0/24 ! -d 10.254.254.0/24 -j SNAT --to 185.28.23.108

If I SNAT to 10.254.254.1, it comes back. But I can't understand why it doesn't for the public IP.

[300000]

You mess up iptables so only you can deal with that. Just remember you need ip forward and NAT at both server and client as they become node for each point of connection. Make sure name of certificate is the same file name in CDD folder if not server cant add correctly route. Let server add route by itself not by you

You cant use site to site with this one. Remove it and try again

push "redirect-gateway def1 bypass-dhcp"

[thalesmaoa]

You are entirely correct.

I finally found the source of the problem, but I can't fix it.
How did I found it, I was looking at my monitor for a while, just thinking about what was going on, and suddenly ping start to come.

I was like... wtf?

My config is correct, but why it took so much time ~3min to reply? I think that this is about some DNS problem, which doesn't occur using MASQUERADE.

So here is my final conf:

View OriginalServer Config

local 89.117.32.207
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
topology subnet
server 10.254.254.0 255.255.255.0
push "redirect-gateway def1 bypass-dhcp"
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
push "block-outside-dns"
keepalive 10 120
cipher AES-256-CBC
user nobody
group nogroup
persist-key
persist-tun
verb 3
crl-verify crl.pem
explicit-exit-notify

client-to-client
client-config-dir /etc/openvpn/ccd

push "route 89.117.32.200 255.255.255.0"

# My Home LAN
route 192.168.200.0 255.255.255.0
push "route 192.168.200.0 255.255.255.0"

# Increase performance
sndbuf 512000
rcvbuf 512000
push "sndbuf 512000"
push "rcvbuf 512000"
txqueuelen 2000

View OriginalClient Config

ifconfig-push 10.254.254.100 255.255.255.0
iroute 192.168.200.0 255.255.255.0

Code: Select all

# Server iptables
iptables -I INPUT -p udp --dport 1194 -j ACCEPT
iptables -I FORWARD -s 10.254.254.0/24 -j ACCEPT
iptables -I FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.254.254.0/24 -j MASQUERADE

Code: Select all

# Client IP Tables
sudo iptables -t nat -A POSTROUTING -s 10.254.254.1/24 -d 192.168.200.1/24 -o eth0 -j MASQUERADE

Don't forget to:

Code: Select all

nano /etc/sysctl.conf

Code: Select all

net.ipv4.ip_forward=1
net.ipv6.conf.all.disable_ipv6=1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6=1