Hello, I'm having trouble to achieve the following: have an Internet client cliI reach srvB via gwA (and the routed VPN between vpnA and vpnB servers) - WITHOUT using MASQUERADE of the client IP address - WITHOUT routing lanB (_client_) Internet traffic through the VPN and gwA (but directly through gwB) [code] cliI (anyone) cliA cliA' | | | | | | Internet (static IP) --- (eth0) gwA (br0) --- (lanA) --- (eth0) vpnA (server) | (tun0) | routed | (tun11) Internet (dynamic IP) ------------------------ (eth0) gwB/(lanB)/vpnB (client) | (br0) | srvB[/code] I'm aware I'll have to implement "exotic" routing on lanB to allow cliI traffic to be routed back from srvB through the VPN (I plan to do this with DSCP; iptables + ip route + ip rule). Currently, I'm stuck at having the traffic of cliI go through the VPN: [b]On gwA: traffic is DNAT-ed to srvB[/b] [code]Chain PREROUTING (policy ACCEPT 2629 packets, 140K bytes) 25 1500 DNAT tcp -- eth0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:993 to:srvB[/code] (NOTE: DNAT packets increases at each cliI connection attempt) [code]Chain FORWARD (policy DROP 0 packets, 0 bytes) 62 3720 ACCEPT tcp -- eth0 br0 0.0.0.0/0 srvB tcp dpt:993[/code] (NOTE: ACCEPT packets increases at each cliI connection attempt) [code]11:04:42.643079 IP (tos 0x0, ttl 46, id 58984, offset 0, flags [DF], proto TCP (6), length 60) cliI.35515 > gwA.993: Flags [S], cksum 0x98d5 (correct), seq 2335247159, win 29200, options [mss 1460,sackOK,TS val 838331743 ecr 0,nop,wscale 7], length 0[/code] [b]On vpnA: traffic is routed to tun0[/b] [code]Chain FORWARD (policy DROP 33 packets, 2772 bytes) 42593 7156K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED 556 33456 ACCEPT all -- eth0 tun0 0.0.0.0/0 0.0.0.0/0[/code] (NOTE: ACCEPT packets increases at each cliI connection attempt) [code]11:04:42.643191 IP (tos 0x0, ttl 45, id 58984, offset 0, flags [DF], proto TCP (6), length 60) cliI.35515 > srvB.993: Flags [S], cksum 0x9669 (correct), seq 2335247159, win 29200, options [mss 1460,sackOK,TS val 838331743 ecr 0,nop,wscale 7], length 0 [/code] [code]Feb 22 11:04:14 vpnA [...]: MULTI: Learn: srvB -> vpnB/gwB:58108 [/code] [b]On vpnB: nothing shows up[/b] [code]Chain FORWARD (policy DROP 68 packets, 7235 bytes) 0 0 ACCEPT tcp -- tun11 * 0.0.0.0/0 srvB tcp dpt:993 [/code] (NOTE: DROP packets remain unchanged through cliI connection attempts) Everything works like a charm for clients cliA and cliA'. If I enable MASQUERADE-ing of cliI IP address, everythin works like a charm (but I really don't want to do that; I need to keep cliI IP address untouched for ACLs and logging) From what I'm able to gather, it seems the incoming packet vanishes between vpnA and vpnB (and nothing relevant shows up in the logs). Has anyone an idea what might happen and if OpenVPN requires some configuration setting for this scenario to work ? Thanks and best, Cédric

Please see the HOWTO: [url=http://openvpn.net/index.php/open-source/documentation/howto.html#scope]HOWTO: Expanding the scope of the VPN to include additional machines[/url]

[quote="cdufour"][b]Afterthoughts:[/b] Could it be that cliI packets arriving at tun11 (on gwB/vpnB) are considered "martians", given the fact that the (non-SNAT/MASQUERADE-ed) source IP does not match any segment associated with the VPN ?[/quote] This was it! [b]On gwB/vpnB:[/b] [code]echo 0 > /proc/sys/net/ipv4/conf/tun11/rp_filter[/code] Solves the problem! [code]Chain FORWARD (policy DROP 68 packets, 7235 bytes) 21 1260 ACCEPT tcp -- tun11 * 0.0.0.0/0 srvB tcp dpt:993[/code] Note that [code]echo 2 > /proc/sys/net/ipv4/conf/tun11/rp_filter[/code] will NOT work since the FIB (routing cache) never gets the proper entries to allow passing of the cliI "martian" packets (its a "chicken and egg" problem). I guess that given this scenario (and "upstream" validation performed by various iptables), disabling IP source validation on the OpenVPN tun interface (only) can be considered reasonable (please correct me if I'm wrong). [b]References:[/b] About rp_filter: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt About the FIB: http://linux-ip.net/html/routing-cache.html

Is [b]Client.Internet[/b] connected [u]to the VPN[/u] at [b]vpnA[/b] (server) ?

Access internal service from the Internet via routed VPN

cdufour

Hello,

I'm having trouble to achieve the following: have an Internet client cliI reach srvB via gwA (and the routed VPN between vpnA and vpnB servers)
- WITHOUT using MASQUERADE of the client IP address
- WITHOUT routing lanB (_client_) Internet traffic through the VPN and gwA (but directly through gwB)

 cliI (anyone)                                  cliA             cliA'
   |                                             |                |
   |                                             |                |
Internet (static IP)  --- (eth0) gwA (br0) --- (lanA) --- (eth0) vpnA (server)
                                                                  | (tun0)
                                                                  | routed
                                                                  | (tun11)
Internet (dynamic IP) ------------------------ (eth0) gwB/(lanB)/vpnB (client)
                                                            | (br0)
                                                            |
                                                           srvB

I'm aware I'll have to implement "exotic" routing on lanB to allow cliI traffic to be routed back from srvB through the VPN (I plan to do this with DSCP; iptables + ip route + ip rule).

Currently, I'm stuck at having the traffic of cliI go through the VPN:

On gwA: traffic is DNAT-ed to srvB

Chain PREROUTING (policy ACCEPT 2629 packets, 140K bytes)
   25  1500 DNAT       tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:993 to:srvB

(NOTE: DNAT packets increases at each cliI connection attempt)

Chain FORWARD (policy DROP 0 packets, 0 bytes)
   62  3720 ACCEPT     tcp  --  eth0   br0     0.0.0.0/0            srvB          tcp dpt:993

(NOTE: ACCEPT packets increases at each cliI connection attempt)

11:04:42.643079 IP (tos 0x0, ttl 46, id 58984, offset 0, flags [DF], proto TCP (6), length 60)
    cliI.35515 > gwA.993: Flags [S], cksum 0x98d5 (correct), seq 2335247159, win 29200, options [mss 1460,sackOK,TS val 838331743 ecr 0,nop,wscale 7], length 0

On vpnA: traffic is routed to tun0

Chain FORWARD (policy DROP 33 packets, 2772 bytes)
42593 7156K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
  556 33456 ACCEPT     all  --  eth0   tun0    0.0.0.0/0            0.0.0.0/0

(NOTE: ACCEPT packets increases at each cliI connection attempt)

11:04:42.643191 IP (tos 0x0, ttl 45, id 58984, offset 0, flags [DF], proto TCP (6), length 60)
    cliI.35515 > srvB.993: Flags [S], cksum 0x9669 (correct), seq 2335247159, win 29200, options [mss 1460,sackOK,TS val 838331743 ecr 0,nop,wscale 7], length 0

Feb 22 11:04:14 vpnA [...]: MULTI: Learn: srvB -> vpnB/gwB:58108

On vpnB: nothing shows up

Chain FORWARD (policy DROP 68 packets, 7235 bytes)
    0     0 ACCEPT     tcp  --  tun11  *       0.0.0.0/0            srvB         tcp dpt:993

(NOTE: DROP packets remain unchanged through cliI connection attempts)

Everything works like a charm for clients cliA and cliA'.

If I enable MASQUERADE-ing of cliI IP address, everythin works like a charm (but I really don't want to do that; I need to keep cliI IP address untouched for ACLs and logging)

From what I'm able to gather, it seems the incoming packet vanishes between vpnA and vpnB (and nothing relevant shows up in the logs).

Has anyone an idea what might happen and if OpenVPN requires some configuration setting for this scenario to work ?

Thanks and best,

Cédric

Traffic

Please see the HOWTO:
HOWTO: Expanding the scope of the VPN to include additional machines

cdufour

Hello,

Thanks for the pointer:

Traffic wrotePlease see the HOWTO:
HOWTO: Expanding the scope of the VPN to include additional machines

However, I believe (correct me if I'm wrong) everything in this HOWTO is already in place. The entire lanA can talk to the entire lanB and vice-versa without any problem. It is when the traffic originate from the InternetA (cliI) that the problem arises (even though the traffic goes all the way to the vpnA server); I can't even see the first incoming packet (I'll deal with a valid return path - cf. DSCP - afterwards)

gwA routing:

<lanA>/24 dev br0  proto kernel  scope link  src <gwA>
<vpn>/24 via <vpnA> dev br0
<lanB>/16 via <vpnA> dev br0
default via <InternetA:static>

vpnA server config (OpenVPN 2.3.4; Debian/Jessie):

# Server
mode server
dev tun
proto tcp-server
port 1194

# Network
# ... server
ifconfig <vpn>.1 <vpn>.2
route <vpn> 255.255.255.0
# ... clients: roaming
ifconfig-pool <vpn>.64 <vpn>.251
ifconfig-pool-persist /var/cache/openvpn/ipp.txt
client-config-dir /etc/openvpn/clients.d/
push "route <vpn> 255.255.255.0"
push "route <lanA> 255.255.255.0"
client-to-client
# ... client: <lanB>
route <lanB> 255.255.0.0

lanB CCD config:

# Network
ifconfig-push <vpn>.6 <vpn>.5
iroute <lanB> 255.255.0.0

DEFAULT CCD config:

# Network
push "dhcp-option DOMAIN <domain>"
push "dhcp-option DNS <lanA:DNS>"
push "route <lanB> 255.255.0.0"

gwB routing:

<lanB>/16 dev br0  proto kernel  scope link  src <gwB>
<vpn>.5 dev tun11  proto kernel  scope link  src <vpn>.6 
<vpn>/24 via <vpn>.5 dev tun11 
<lanA>/24 via <vpn>.5 dev tun11 
default via <InternetB:dynamic>

vpnB client config (OpenVPN 2.3.8; AsusWRT/RMerl):

daemon
client
dev tun11
proto tcp-client
remote <gwA:public> 1194
resolv-retry 30
nobind
script-security 2
route-delay 2
route-up vpnrouting.sh
route-pre-down vpnrouting.sh

(NOTE: I verified the logs; given this setup, the vpnrouting.sh script does nothing)

Afterthoughts:
Could it be that cliI packets arriving at tun11 (on gwB/vpnB) are considered "martians", given the fact that the (non-SNAT/MASQUERADE-ed) source IP does not match any segment associated with the VPN ?

cdufour

cdufour wroteAfterthoughts:
Could it be that cliI packets arriving at tun11 (on gwB/vpnB) are considered "martians", given the fact that the (non-SNAT/MASQUERADE-ed) source IP does not match any segment associated with the VPN ?

This was it!

On gwB/vpnB:

echo 0 > /proc/sys/net/ipv4/conf/tun11/rp_filter

Solves the problem!

Chain FORWARD (policy DROP 68 packets, 7235 bytes)
   21  1260 ACCEPT     tcp  --  tun11  *       0.0.0.0/0            srvB         tcp dpt:993

Note that

echo 2 > /proc/sys/net/ipv4/conf/tun11/rp_filter

will NOT work since the FIB (routing cache) never gets the proper entries to allow passing of the cliI "martian" packets (its a "chicken and egg" problem).

I guess that given this scenario (and "upstream" validation performed by various iptables), disabling IP source validation on the OpenVPN tun interface (only) can be considered reasonable (please correct me if I'm wrong).

References:
About rp_filter: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
About the FIB: http://linux-ip.net/html/routing-cache.html

Traffic

Is Client.Internet connected to the VPN at vpnA (server) ?

cdufour

Traffic wroteIs Client.Internet connected to the VPN at vpnA (server) ?

No. I agree "client" is misleading here. cliI is a public service client, not a VPN client (cliA' is a VPN client).

The idea of the setup is to allow srvB (public service) to be reached via the static public IP address available on lanA site.

cdufour

If ever someone stumbles on this post and attempts to do the same trick, below are the missing parts.

Start with http://openvpn.net/index.php/open-source/documentation/howto.html#scope
(and make sure everything works as expected before proceeding)

On gwB/vpnB:

Accept "martians" packets on the VPN interface:

echo 0 > /proc/sys/net/ipv4/conf/tun11/rp_filter

Tag the traffic originating from the VPN - using DSCP value 12 (class AF12) or any that is not used on the local network - and make sure it is routed back to the VPN (and make sure DSCP value is reset to 0 when leaving the local network):

# Routing
ip route flush table 12
ip route add default dev tun11 table 12
ip rule add fwmark 12 table 12

# DSCP
iptables -t mangle -A PREROUTING -i tun11 -j DSCP --set-dscp 12
iptables -t mangle -A PREROUTING -i br0 -m dscp --dscp 12 -j MARK --set-mark 12
iptables -t mangle -A POSTROUTING -o ! br0 -j DSCP --set-dscp 0

On srvB:

Make sure the DSCP tag present on the INPUT traffic is set back on the OUTPUT traffic:

iptables -t mangle -A INPUT -m dscp --dscp 12 -j CONNMARK --set-mark 12
iptables -t mangle -A OUTPUT -m connmark --mark 12 -j DSCP --set-dscp 12

PS: table ID, mark and connmark values are all match the DSCP value (12) as a convenience (they can be any other value as long as the entire setup is consistent)