sending packets whose destination address matches an IP on another interface on the peer

Scripts with setup, destroy, and modify routing tables and firewall rulesets for client connections.

Moderators: TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech

Post Reply
jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

sending packets whose destination address matches an IP on another interface on the peer

Post by jimays » Mon Mar 05, 2018 7:41 pm

Hello,
This is my first posting here, gratitude for all the good posts.

I have a peer-to-peer setup which is functioning great.
I use the peer addresses as via addresses for routing to cross-network IPs.

For various reasons relative to correctly handling hairpin NAT, I have SNAT'd a packet into the tunnel so that it arrives at its
destination with a source address matching the IP address of another interface on the same source machine.

The peer receives the packet correctly, and responds into the tunnel, visible by tcpdump on the peer.

The tcpdump on the source side has yet to ever show a return packet.
This is a packet whose DST= matches the IP on another interface on the destination machine,
that is on the destination machine for the return packet, which is the original source machine.

192.168.56.157 -> 96.64.59.106:64008 original packet from internal source arriving at machine with vpn tunnel
nat prerouting dnat(96.64.159.106:64008 to 192.168.56.248:8024)
nat postrouting snat(192.168.56.157->96.64.159.105)
routes into tunnel due to packet mark and special routing table
96.64.159.105 -> 192.168.56.248 at peer arrives like this, forwards to internal destination
92.168.56.248 -> 96.64.159.105 response from internal arrives at peer
routes into tunnel due to packet mark and special routing table
tcpdump shows the reply packet entering the tunnel, heading back through to tunnel to the original source side
and where o where can I find evidence of this reply packet in the tcpdump on that original source side?

Here is the iptables TRACE for the source packet coming arriving from its internal source and being forwarded into the tunnel.

0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: raw:RW_GENERAL_TRACING:rule:8 IN=vzbrI56 MAC=fe:3f:44:89:ae:ab:02:93:b8:44:86:2d:08:00 SRC=192.168.56.157 DST=96.64.159.106 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=1166 DF PROTO=TCP SPT=42252 DPT=64008 SEQ=1100496672 ACK=0 WINDOW=29200 SYN
...
0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: nat:SN_EPHEM_A_7_3:rule:2 OUT=tunvs7s2vc7s1 SRC=192.168.56.157 DST=192.168.56.248 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=1166 DF PROTO=TCP SPT=42252 DPT=8024 SEQ=1100496672 ACK=0 WINDOW=29200 SYN mark=1073743619

The nat rule changes the source address to 96.64.159.105, which is an IP owned by an ethernet interface on the source machine.

iptables -A SN_EPHEM_A_7_3 -s 192.168.56.0/23 -j SNAT --to-source 96.64.159.105

Here is the trace for the source packet emerging at the peer and being forwarded to its internal destination.

0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: raw:RW_GENERAL_TRACING:rule:8 IN=tunvc7s1vs7s2 SRC=96.64.159.105 DST=192.168.56.248 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=1166 DF PROTO=TCP SPT=42252 DPT=8024 SEQ=1100496672 ACK=0 WINDOW=29200 SYN
...
0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: nat:POSTROUTING:policy:6 OUT=vzbrI56 SRC=96.64.159.105 DST=192.168.56.248 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=1166 DF PROTO=TCP SPT=42252 DPT=8024 SEQ=1100496672 ACK=0 WINDOW=29200 SYN mark=1073741824

Here is the return packet coming from the internal destination back to the peer and being forwarded back to the source.

0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: raw:RW_GENERAL_TRACING:rule:8 IN=vzbrI56 MAC=fe:d4:d2:c6:3f:6c:ae:08:42:f8:f8:26:08:00 SRC=192.168.56.248 DST=96.64.159.105 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=8024 DPT=42252 SEQ=4232934541 ACK=1100496673 WINDOW=28960 ACK SYN
...
0 6 - 05/Mar/2018:12:17:58 -0700 TRACE: mangle:POSTROUTING:policy:5 OUT=tunvc7s1vs7s2 SRC=192.168.56.248 DST=96.64.159.105 LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=8024 DPT=42252 SEQ=4232934541 ACK=1100496673 WINDOW=28960 ACK SYN mark=18180

Here is the tcpdump of the return packet entering the tunnel.

17:19:00.508904 IP odoo7-248.idsp56.net.8024 > 96-64-159-105-static.hfc.comcastbusiness.net.40004: Flags [S.], seq 782673324, ack 2663188677, win 28960, options [mss 1460,sackOK,TS val 1538743048 ecr 1922512830,nop,wscale 7], length 0

I have the same iptables trace rule on both sides, so I expect to see the packet emerge, ready for dnat.

iptables -t raw -I RW_GENERAL_TRACING 4 -p tcp --sport 8024 -s 192.168.56.248 -j TRACE

I also ran a full wildcard tcpdump on the receiving side and never see the packet.
This is a packet from 192.168.56.248 to 96.64.159.105, through a tunnel to a target where 96.64.159.105 is an IP on another interface.

Can anyone offer any insight as to whether this is the expected behavior of openvpn in this case?
Can anyone suggest how I can see my packet? I do have more trace output, tried to be minimal for this first posting.
I am willing to adjust a local copy of the C code, though I would prefer a config-based solution.

Gratitude.

jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

Why would any TCP packet only appear on one side of a peer-to-peer in tcpdump and iptables trace?

Post by jimays » Fri Mar 09, 2018 12:05 am

I restate my question more concisely:

Why would any TCP packet only appear on one side of a peer-to-peer in tcpdump and iptables trace?

I would be very grateful for some enlightenment.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Fri Mar 09, 2018 12:10 am


jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by jimays » Sat Mar 17, 2018 4:35 am

Hello openvpn seers.
I am attempting to be more conformant to 'How to Post'...
and am posting a simpler case that illustrates the same situation.

SERVER@10.250.60.121 with tunnel endpoint 10.249.58.57
CLIENT@10.250.60.130 with tunnel endpoint 10.249.57.58

server

dev tunvs7s2vc7s1
local 10.250.60.121
port 1194
proto udp
remote 10.250.60.130 1194
float

topology p2p
ifconfig 10.249.58.57 10.249.57.58
script-security 2
route-noexec
#route-up "/tru/mnt/realm-net-tools/bin/realm-net-routing --run tunvs7s2vc7s1"

tls-server
ping-restart 120

ping 10
persist-key
persist-tun

ca /etc/openvpn/realm-certs/^inter-dimensional-space-port.vpn.crt
cert /etc/openvpn/realm-certs/vpn-link-server/^inter-dimensional-space-port.net^vpn-vs7s2vc7s1.inter-dimensional-space-port.net.crt
key /etc/openvpn/realm-certs/vpn-link-server/^inter-dimensional-space-port.net^vpn-vs7s2vc7s1.inter-dimensional-space-port.net.key
dh /etc/openvpn/realm-certs/dh2048.pem
cipher AES-256-CBC

verb 3
status /dev/null

;log openvpn.log
;log-append openvpn.log
;mute 20

user nobody
group nogroup


client

dev tunvc7s1vs7s2
local 10.250.60.130
port 1194
proto udp
remote 10.250.60.121 1194
float

topology p2p
ifconfig 10.249.57.58 10.249.58.57
script-security 2
route-noexec
#route-up "/etc/realm-net/moc7-41.route-up.sh"

tls-client
ping-restart 180

ping 10
persist-key
persist-tun

ca /etc/openvpn/realm-certs/^inter-dimensional-space-port.vpn.crt
cert /etc/openvpn/realm-certs/vpn-link-client/^inter-dimensional-space-port.net^vpn-vc7s1vs7s2.inter-dimensional-space-port.net.crt
key /etc/openvpn/realm-certs/vpn-link-client/^inter-dimensional-space-port.net^vpn-vc7s1vs7s2.inter-dimensional-space-port.net.key

cipher AES-256-CBC

verb 3
status /dev/null

;log openvpn.log
;log-append openvpn.log
;mute 20

user nobody
group nogroup


On the server:

Code: Select all

service openvpn@vpn-vs7s2vc7s1 start
iptables -t raw -A OUTPUT -d 10.250.60.130 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.130 -j MARK --set-mark 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.57.58 dev tunvs7s2vc7s1
tcpdump -i tunvs7s2vc7s1 # watch server side of tunnel
On the client:

Code: Select all

service openvpn@vpn-vc7s1vs7s2 start
iptables -t raw -A OUTPUT -d 10.250.60.121 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.121 -j MARK --set-mark 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.58.57 dev tunvs7s2vc7s1
tcpdump -i tunvs7s2vc7s1 # watch client side of tunnel
From another window on server:

Code: Select all

ping -I 10.249.58.57 10.249.57.58 # successful, shows in both tcpdump displays and both iptables traces
ping 10.250.60.130 # shows only in server tcpdump and iptables
This packet has src=10.250.60.121, dst=10.250.60.130, and that dst is an IP on a non-tunnel interface on the destination machine.
I am thinking openvpn is by default reluctant to source into a box a packet that would appear to come from one of its own other interfaces.

My question is how can I recover this packet on the peer?
Gratitude.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Sat Mar 17, 2018 1:59 pm

jimays wrote:
Sat Mar 17, 2018 4:35 am
server
local 10.250.60.121
jimays wrote:
Sat Mar 17, 2018 4:35 am
client
local 10.250.60.130
jimays wrote:
Sat Mar 17, 2018 4:35 am
ping 10.250.60.130 # shows only in server tcpdump and iptables
Assuming /24 then such a packet will not be routed via the tunnel ...

jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by jimays » Sat Mar 17, 2018 5:37 pm

Kindly excuse the typo in the tunnel names in my last post.
These commands here were pasted directly from my test scripts.

experiment-server.sh:

Code: Select all

service openvpn@vpn-vs7s2vc7s1 start
iptables -t raw -A OUTPUT -d 10.250.60.130 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.130 -j MARK --set-mark 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.57.58 dev tunvs7s2vc7s1
tcpdump -i tunvs7s2vc7s1 # watch server side of tunnel
experiment-client.sh:

Code: Select all

service openvpn@vpn-vc7s1vs7s2 start
iptables -t raw -A OUTPUT -d 10.250.60.121 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.121 -j MARK --set-mark 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.58.57 dev tunvc7s1vs7s2
tcpdump -i tunvc7s1vs7s2 # watch client side of tunnel
running on the server: ping -I 10.249.58.57 10.249.57.58

tcpdump on server:

Code: Select all

11:29:43.333416 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 1, length 64
11:29:43.334120 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 1, length 64
11:29:44.335522 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 2, length 64
11:29:44.336075 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 2, length 64
11:29:45.359535 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 3, length 64
11:29:45.360066 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 3, length 64
...
tcpdump on client:

Code: Select all

11:29:42.965634 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 1, length 64
11:29:42.965661 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 1, length 64
11:29:43.967520 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 2, length 64
11:29:43.967541 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 2, length 64
11:29:44.991500 IP 10.249.58.57 > 10.249.57.58: ICMP echo request, id 7988, seq 3, length 64
11:29:44.991519 IP 10.249.57.58 > 10.249.58.57: ICMP echo reply, id 7988, seq 3, length 64
...
running on the server: ping 10.250.60.130

tcpdump on server, which I believe is showing packets actually entering the tunnel on the server side:

Code: Select all

11:33:17.400671 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 14193, seq 1, length 64
11:33:18.415545 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 14193, seq 2, length 64
11:33:19.439537 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 14193, seq 3, length 64
...
tcpdump on client, which I am expecting to show all packets entering from the server:

Code: Select all

(empty, and why would that be, and how can we see these packets?)


Gratitude for the assistance.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Sat Mar 17, 2018 5:55 pm

I can see what you are trying to do ..
I do not understand why you are trying to do it ..

For me this has nothing to do with openvpn ..

* Thread has been moved *

As a side note: I presume IP forwarding is enabled on both hosts.

EDIT: perhaps your routing tables will have the information you need ..

jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

Am I correct in thinking the client and server tcpdump streams on a p2p should be identical regardless of any routing ru

Post by jimays » Sun Mar 18, 2018 11:05 am

Good idea to check the packet forwarding. I also checked reverse-packet filtering.

Code: Select all

root@idspgw7-a:/etc/openvpn# cat /proc/sys/net/ipv4/ip_forward
1
root@idspgw7-a:/etc/openvpn# cat /proc/sys/net/ipv4/conf/all/rp_filter
0
root@idspgw7-a:/etc/openvpn# cat /proc/sys/net/ipv4/conf/enp1s0/rp_filter
0
root@idspgw7-a:/etc/openvpn# cat /proc/sys/net/ipv4/conf/tunvc7s1vs7s2/rp_filter
0
root@ahau1:/etc/openvpn# cat /proc/sys/net/ipv4/ip_forward
1
root@ahau1:/etc/openvpn# cat /proc/sys/net/ipv4/conf/all/rp_filter
0
root@ahau1:/etc/openvpn# cat /proc/sys/net/ipv4/conf/eth0/rp_filter
0
root@ahau1:/etc/openvpn# cat /proc/sys/net/ipv4/conf/tunvs7s2vc7s1/rp_filter
0
The routing rules are set such that non-port-1194 traffic between the two external endpoints is re-routed through the tunnel.
The reason is related to how I am handling hairpin nat within a larger connected network, which I could discuss if thought helpful.
I have isolated the case here down to two IP addresses in a reproducable case just to analyze the openvpn interaction.

Since the packet enters the tunnel, it must have had a route out of the server to the client.
I would expect to see the packet exit the tunnel on the client side, and for the packet to then enter netfilter on the client, like for other packets.

For example, if I have a machine with eth0=1.1.1.1/24 and eth1=2.2.2.2/24, and I send in a packet from another host 1.1.1.2/24 via 1.1.1.1,
and that packet has src=3.3.3.3/dst=2.2.2.2, then the packet still enters netfilter, the trace shows, and with proper rules the packet can be routed
and enters the INPUT or FORWARD chains.

I would expect the same behavior from a tunnel interface as for eth0 in the example.

In my experiment, I see no packet exiting the tunnel on the client side and no packet in the client iptables -t raw -j TRACE.
I run tcpdump on both sides and only see the packet on the source side, so it seems openvpn is not passing this packet.

*** Am I correct in thinking the client and server tcpdump streams on a p2p should be identical regardless of any routing rules on either side?

For reference, the routing rules are the default at-boot rules as augmented by starting openvpn and running the ip rule and ip route commands.
Here is the final setup for the experiment:

On the server:

Code: Select all

root@ahau1:/etc/openvpn# ip ru
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default 

root@ahau1:/etc/openvpn# ip r
default via 10.250.60.254 dev eth0 onlink 
10.249.57.58 dev tunvs7s2vc7s1 proto kernel scope link src 10.249.58.57 
10.250.60.0/24 dev eth0 proto kernel scope link src 10.250.60.121 

root@ahau1:/etc/openvpn# ip r s t 1
default via 10.249.57.58 dev tunvs7s2vc7s1

root@ahau1:/etc/openvpn# iptables -t mangle -nvL OUTPUT
Chain OUTPUT (policy ACCEPT 7670 packets, 829K bytes)
 pkts bytes target     prot opt in     out     source               destination         
 6412  680K RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:1194
   75  6300 MARK       all  --  *      *       0.0.0.0/0            10.250.60.130        MARK set 0x1
On the client:

Code: Select all

root@idspgw7-a:/etc/openvpn# ip ru
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default 

root@idspgw7-a:/etc/openvpn# ip r
default via 10.250.60.254 dev enp1s0 onlink 
10.249.58.57 dev tunvc7s1vs7s2  proto kernel  scope link  src 10.249.57.58 
10.250.60.0/24 dev enp1s0  proto kernel  scope link  src 10.250.60.130 
169.254.0.0/16 dev enp1s0  scope link  metric 1000 

root@idspgw7-a:/etc/openvpn# ip r s t 1
default via 10.249.58.57 dev tunvc7s1vs7s2

root@idspgw7-a:/etc/openvpn# iptables -t mangle -nvL OUTPUT
Chain OUTPUT (policy ACCEPT 366K packets, 28M bytes)
 pkts bytes target     prot opt in     out     source               destination         
 6405  677K RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:1194
    0     0 MARK       all  --  *      *       0.0.0.0/0            10.250.60.121        MARK set 0x1

root@idspgw7-a:/etc/openvpn# iptables -t raw -nvL  # tracing to search for all ICMP packets
Chain PREROUTING (policy ACCEPT 334 packets, 26095 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  160 14080 TRACE      icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
Chain OUTPUT (policy ACCEPT 267 packets, 29993 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   80  7040 TRACE      icmp --  *      *       0.0.0.0/0            0.0.0.0/0           
Gratitude.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Sun Mar 18, 2018 8:50 pm

jimays wrote:
Sun Mar 18, 2018 11:05 am
Since the packet enters the tunnel, it must have had a route out of the server to the client.
I would expect to see the packet exit the tunnel on the client side, and for the packet to then enter netfilter on the client, like for other packets
You have --tls-server/client mixed with --topology p2p .. I forgot to mention that you will probably need an --iroute. (The 'i' is deliberate)

See --iroute in The Manual v24x

Also, this explains in more detail:
HOWTO: Expanding the scope of the VPN to include additional machines

Hopefully, that should fix it :geek:

jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

asymmetric behavior of intended symmetric peer-to-peer network, dropping some server-to-client packets

Post by jimays » Mon Mar 19, 2018 1:11 am

All code snippets from previous posts can be ignored. I repost all here, simplified to just a few settings, free of TLS, no scrollbars required.
Once I find a solution, I would be happy to re-phrase the question one more time for helping the bulletin board with a good topic name....

*** I continued to verify the symmetry of my test config and found that it is only server-to-client packets that do not emerge,
*** so my initial somewhat dubious theory about the situation having to do with addresses on other interfaces is hereby debunked.
Client-to-server rides just fine through the tunnel and is received by the server even with that dst address on another intrerface.

On your suggestion, I thought I would try adding "iroute 0.0.0.0 0.0.0.0" to the server config. From the man page:
"This option must be specified either in a client instance config file using --client-config-dir or dynamically generated using a --client-connect script."
Both --client-config-dir and --client-connect caused a message saying the option requires --mode server, and I am using --mode p2p.

It makes sense the situation could be due to something similar to the iroute mechanism. I wonder what could affect p2p mode.
I am considering trying mode server with only one client, still I specifically selected p2p for its simplicity, so I would prefer to stay with p2p.
I really like using only two IP addresses and setting up the via routes directly.

Here is the symmetric simplified config:

SERVER

dev tuntestserver
local 10.250.60.121
port 1194
proto udp
remote 10.250.60.130 1194
topology p2p
ifconfig 10.249.58.57 10.249.57.58
route-noexec


CLIENT

dev tuntestclient
local 10.250.60.130
port 1194
proto udp
remote 10.250.60.121 1194
topology p2p
ifconfig 10.249.57.58 10.249.58.57
route-noexec


experiment-server.sh

Code: Select all

service openvpn@vpn-testserver restart
iptables -t raw -F ; iptables -t mangle -F
iptables -t raw -A OUTPUT -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t raw -A PREROUTING -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.130 -j MARK --set-mark 1
ip rule delete priority 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.57.58 dev tuntestserver
ip route show
ip route show table 1
ip rule show
tcpdump -i tuntestserver &
experiment-client.sh

Code: Select all

service openvpn@vpn-testserver restart
iptables -t raw -F ; iptables -t mangle -F
iptables -t raw -A OUTPUT -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t raw -A PREROUTING -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.130 -j MARK --set-mark 1
ip rule delete priority 1
ip rule add priority 1 fwmark 1 lookup 1
ip route add table 1 default via 10.249.57.58 dev tuntestserver
ip route show
ip route show table 1
ip rule show
tcpdump -i tuntestserver &
bash -x experiment-server.sh # excerpted

Code: Select all

+ ip route show
default via 10.250.60.254 dev eth0 onlink 
10.249.57.58 dev tuntestserver proto kernel scope link src 10.249.58.57 
10.250.60.0/24 dev eth0 proto kernel scope link src 10.250.60.121 
+ ip route show table 1
default via 10.249.57.58 dev tuntestserver 
+ ip rule show
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default 
bash -x experiment-client.sh # excerpted

Code: Select all

+ ip route show
default via 10.250.60.254 dev enp1s0 onlink 
10.249.58.57 dev tuntestclient  proto kernel  scope link  src 10.249.57.58 
10.250.60.0/24 dev enp1s0  proto kernel  scope link  src 10.250.60.130 
+ ip route show table 1
default via 10.249.58.57 dev tuntestclient 
+ ip rule show
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default
Ping 10.250.60.130 from server to client, tcpdump on the server:

Code: Select all

18:05:25.675008 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 5220, seq 1, length 64
18:05:26.703543 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 5220, seq 2, length 64
18:05:27.727543 IP 10.250.60.121 > 10.250.60.130: ICMP echo request, id 5220, seq 3, length 64
Ping 10.250.60.130 from server to client, tcpdump on the client:

Code: Select all

(empty)
Ping 10.250.60.121 from client to server, tcpdump on the client:

Code: Select all

18:06:20.172161 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 2, length 64
18:06:21.172174 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 3, length 64
18:06:22.172156 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 4, length 64
Ping 10.250.60.121 from client to server, tcpdump on the server, turning around as expected:

Code: Select all

18:06:19.113623 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 1, length 64
18:06:19.113698 IP 10.250.60.121 > 10.250.60.130: ICMP echo reply, id 3140, seq 1, length 64
18:06:20.112836 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 2, length 64
18:06:20.112894 IP 10.250.60.121 > 10.250.60.130: ICMP echo reply, id 3140, seq 2, length 64
18:06:21.112836 IP 10.250.60.130 > 10.250.60.121: ICMP echo request, id 3140, seq 3, length 64
18:06:21.112909 IP 10.250.60.121 > 10.250.60.130: ICMP echo reply, id 3140, seq 3, length 64
I am thinking in the client-to-server case, the replies never make it back to the client
likely for the same reason the server-to-client source packets don't make it.

iptables trace for ping 10.250.60.121 leaving the client:

Code: Select all

Mar 18 18:29:49 idspgw7-a kernel: [112987.641635] TRACE: raw:OUTPUT:policy:2 IN= OUT=enp1s0 SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1 UID=0 GID=0
Mar 18 18:29:49 idspgw7-a kernel: [112987.641645] TRACE: mangle:OUTPUT:rule:2 IN= OUT=enp1s0 SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1 UID=0 GID=0 
Mar 18 18:29:49 idspgw7-a kernel: [112987.641658] TRACE: mangle:OUTPUT:policy:3 IN= OUT=enp1s0 SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1 UID=0 GID=0 MARK=0x1
Mar 18 18:29:49 idspgw7-a kernel: [112987.641721] TRACE: mangle:POSTROUTING:policy:1 IN= OUT=tuntestclient SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1 UID=0 GID=0 MARK=0x1
iptables trace for ping 10.250.60.121 correctly turning around at server:

Code: Select all

0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: raw:PREROUTING:policy:2 IN=tuntestserver SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: mangle:PREROUTING:policy:1 IN=tuntestserver SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: mangle:INPUT:policy:1 IN=tuntestserver SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: filter:INPUT:policy:1 IN=tuntestserver SRC=10.250.60.130 DST=10.250.60.121 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=25479 DF PROTO=ICMP TYPE=8 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: raw:OUTPUT:policy:2 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=65341 PROTO=ICMP TYPE=0 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: mangle:OUTPUT:rule:2 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=65341 PROTO=ICMP TYPE=0 CODE=0 ID=3153 SEQ=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: mangle:OUTPUT:policy:3 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=65341 PROTO=ICMP TYPE=0 CODE=0 ID=3153 SEQ=1 mark=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: filter:OUTPUT:policy:1 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=65341 PROTO=ICMP TYPE=0 CODE=0 ID=3153 SEQ=1 mark=1
0 6 - 18/Mar/2018:18:29:49 -0600 TRACE: mangle:POSTROUTING:policy:1 OUT=tuntestserver SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=65341 PROTO=ICMP TYPE=0 CODE=0 ID=3153 SEQ=1 mark=1
iptables trace for ping 10.250.60.130 leaving the server, symmetric to the client-to-server case:

Code: Select all

0 6 - 18/Mar/2018:18:35:03 -0600 TRACE: raw:OUTPUT:policy:2 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=6785 DF PROTO=ICMP TYPE=8 CODE=0 ID=17006 SEQ=1
0 6 - 18/Mar/2018:18:35:03 -0600 TRACE: mangle:OUTPUT:rule:2 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=6785 DF PROTO=ICMP TYPE=8 CODE=0 ID=17006 SEQ=1
0 6 - 18/Mar/2018:18:35:03 -0600 TRACE: mangle:OUTPUT:policy:3 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=6785 DF PROTO=ICMP TYPE=8 CODE=0 ID=17006 SEQ=1 mark=1
0 6 - 18/Mar/2018:18:35:03 -0600 TRACE: filter:OUTPUT:policy:1 OUT=eth0 SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=6785 DF PROTO=ICMP TYPE=8 CODE=0 ID=17006 SEQ=1 mark=1
0 6 - 18/Mar/2018:18:35:03 -0600 TRACE: mangle:POSTROUTING:policy:1 OUT=tuntestserver SRC=10.250.60.121 DST=10.250.60.130 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=6785 DF PROTO=ICMP TYPE=8 CODE=0 ID=17006 SEQ=1 mark=1
iptables trace for ping 10.250.60.130 "arriving" at client:

Code: Select all

(empty)
To be complete, so this post can stand by itself, per your suggestion I have verified forwarding is enabled,
and I have also disabled rp filtering on both machines.

Code: Select all

echo 1 > /proc/sys/net/ipv4/ip_forward
for f in $(find /proc/sys -name rp_filter) ; do echo 0 > $f ; done
Gratitude and Regards :)

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Mon Mar 19, 2018 2:14 am

There is a simple solution .. ;)

https://openvpn.net/index.php/open-sour ... howto.html

Make sure it works .. then apply your firewall and routing.

jimays
OpenVpn Newbie
Posts: 7
Joined: Fri Mar 02, 2018 9:21 pm

asymmetric tcpdumps in p2p in recent openvpn versions

Post by jimays » Thu Mar 22, 2018 2:27 am

Hello,
The plot thickens.
While augmenting my test scenario on your request, I decided to switch computers in the lab, and the pings started happening. Hmmm, said I.

The bottom line is if I run openvpn-2.3.10-1ubuntu2.1 on both sides my pings succeed.
I had openvpn-2.4.0-6+deb9u2 on one side when I had the asymmetric behavior.

Real good news for me and our members is that on my production Debian 9 servers I copied over an amd64 openvpn-2.3.10-1ubuntu2.1 binary
onto both sides, and now my hairpin NAT setup is running smoothly.

So says I, what next?

I went back to my experiment machines and built openvpn-2.5_git on both machines, one i386, one amd64.
I can confirm that my experiment as described in my previous post shows the asymmetric tcpdump behavior in 2.5_git,
meaning I see certain packets only on 1-ot-of-2 tcpdumps.

I switch to 2.3.10 on both sides, restart the tunnel, and the pings happen.

I also noticed in my last post I sent the content of experiment-server.sh twice so I repeat all files below.
These files are now using 10.250.60.120 and 10.250.60.130 as the tunnel addresses which is slightly more symmetrical.
I also added 10.250.120.120 on a bridge on the client and 10.250.130.130 on a bridge on the server to prove general routing.

SERVER

dev tuntestserver
local 10.250.60.120
port 1194
proto udp
remote 10.250.60.130 1194
topology p2p
ifconfig 10.249.58.57 10.249.57.58
route-noexec


CLIENT

dev tuntestclient
local 10.250.60.130
port 1194
proto udp
remote 10.250.60.120 1194
topology p2p
ifconfig 10.249.57.58 10.249.58.57
route-noexec


/etc/network/interfaces server:

Code: Select all

auto eth0
iface eth0 inet static
  address 10.250.60.120
  netmask 255.255.255.0
  gateway 10.250.60.254
  post-up ip link add brserver type bridge
  post-up ip link set brserver up
  post-up ip address add 10.250.120.120/24 dev brserver
/etc/network/interfaces client:

Code: Select all

auto enp1s0
iface enp1s0 inet static
  address 10.250.60.130
  netmask 255.255.255.0
  gateway 10.250.60.254
  post-up ip link add brclient type bridge
  post-up ip link set brclient up
  post-up ip address add 10.250.130.130/24 dev brclient
experiment-server.sh

Code: Select all

service openvpn@vpn-testserver restart
iptables -t raw -F ; iptables -t mangle -F
iptables -t raw -A OUTPUT -p icmp -d 10.250.0.0/16 -j TRACE
iptables -t raw -A PREROUTING -p icmp -d 10.250.0.0/16 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.130 -j MARK --set-mark 1
ip rule delete priority 1 ; ip rule add priority 1 fwmark 1 lookup 1
ip route replace table 1 default via 10.249.57.58 dev tuntestserver
ip route replace 10.250.130.0/24 dev tuntestserver src 10.250.120.120
ip route show
ip route show table 1
ip rule show
tcpdump -i tuntestserver &
experiment-client.sh

Code: Select all

service openvpn@vpn-testclient restart
iptables -t raw -F ; iptables -t mangle -F
iptables -t raw -A OUTPUT -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t raw -A PREROUTING -p icmp -d 10.250.60.0/24 -j TRACE
iptables -t mangle -A OUTPUT -p udp --dport 1194 -j RETURN
iptables -t mangle -A OUTPUT -d 10.250.60.120/31 -j MARK --set-mark 1
ip rule delete priority 1 ; ip rule add priority 1 fwmark 1 lookup 1
ip route replace table 1 default via 10.249.58.57 dev tuntestclient
ip route replace 10.250.120.0/24 dev tuntestclient src 10.250.130.130
ip route show
ip route show table 1
ip rule show
tcpdump -i tuntestclient &
bash -x experiment-server.sh # excerpted

Code: Select all

+ ip route show
default via 10.250.60.254 dev eth0 onlink 
10.249.57.58 dev tuntestserver proto kernel scope link src 10.249.58.57 
10.250.60.0/24 dev eth0 proto kernel scope link src 10.250.60.120 
10.250.120.0/24 dev brserver proto kernel scope link src 10.250.120.120 
10.250.130.0/24 dev tuntestserver scope link src 10.250.120.120 
+ ip route show table 1
default via 10.249.57.58 dev tuntestserver 
+ ip rule show
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default
bash -x experiment-client.sh # excerpted

Code: Select all

+ ip route show
default via 10.250.60.254 dev enp1s0 onlink 
10.249.58.57 dev tuntestclient  proto kernel  scope link  src 10.249.57.58 
10.250.60.0/24 dev enp1s0  proto kernel  scope link  src 10.250.60.130 
10.250.120.0/24 dev tuntestclient  scope link  src 10.250.130.130 
10.250.130.0/24 dev brclient  proto kernel  scope link  src 10.250.130.130 
+ ip route show table 1
default via 10.249.58.57 dev tuntestclient 
+ ip rule show
0:	from all lookup local 
1:	from all fwmark 0x1 lookup 1 
32766:	from all lookup main 
32767:	from all lookup default 
Now the fun.

With 2.3.10 on both sides, prove correct ping from client to server bridge:

Code: Select all

root@idspgw7-a:~# ping 10.250.120.120
PING 10.250.120.120 (10.250.120.120) 56(84) bytes of data.
64 bytes from 10.250.120.120: icmp_seq=1 ttl=64 time=0.965 ms
64 bytes from 10.250.120.120: icmp_seq=2 ttl=64 time=0.714 ms
tcpdump on client tunnel interface:

Code: Select all

19:37:03.579608 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2013, seq 1, length 64
19:37:03.580549 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2013, seq 1, length 64
19:37:04.580673 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2013, seq 2, length 64
19:37:04.581375 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2013, seq 2, length 64
tcpdump on server tunnel interface:

Code: Select all

19:37:03.611313 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2013, seq 1, length 64
19:37:03.611402 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2013, seq 1, length 64
19:37:04.612448 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2013, seq 2, length 64
19:37:04.612530 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2013, seq 2, length 64
With 2.3.10 on both sides, prove correct ping from server to client bridge:

Code: Select all

root@ahau1:~# ping 10.250.130.130
PING 10.250.130.130 (10.250.130.130) 56(84) bytes of data.
64 bytes from 10.250.130.130: icmp_seq=1 ttl=64 time=0.909 ms
64 bytes from 10.250.130.130: icmp_seq=2 ttl=64 time=0.585 ms
tcpdump on server tunnel interface:

Code: Select all

19:41:39.600735 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 6922, seq 1, length 64
19:41:39.601578 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 6922, seq 1, length 64
19:41:40.601730 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 6922, seq 2, length 64
19:41:40.602290 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 6922, seq 2, length 64
tcpdump on client tunnel interface:

Code: Select all

19:41:39.570693 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 6922, seq 1, length 64
19:41:39.570722 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 6922, seq 1, length 64
19:41:40.571449 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 6922, seq 2, length 64
19:41:40.571467 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 6922, seq 2, length 64
Now run the test, first from server to client, with openvpn 2.3.10, succeeds:

Code: Select all

root@ahau1:~# ping 10.250.60.130
PING 10.250.60.130 (10.250.60.130) 56(84) bytes of data.
64 bytes from 10.250.60.130: icmp_seq=1 ttl=64 time=0.700 ms
64 bytes from 10.250.60.130: icmp_seq=2 ttl=64 time=0.991 ms
tcpdump on the server, packets correctly routed through the tunnel:

Code: Select all

19:44:52.104097 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 11500, seq 1, length 64
19:44:52.104739 IP 10.250.60.130 > 10.250.60.120: ICMP echo reply, id 11500, seq 1, length 64
19:44:53.111335 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 11500, seq 2, length 64
19:44:53.112249 IP 10.250.60.130 > 10.250.60.120: ICMP echo reply, id 11500, seq 2, length 64
tcpdump on the client:

Code: Select all

19:44:52.066904 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 11500, seq 1, length 64
19:44:52.066975 IP 10.250.60.130 > 10.250.60.120: ICMP echo reply, id 11500, seq 1, length 64
19:44:53.074138 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 11500, seq 2, length 64
19:44:53.074189 IP 10.250.60.130 > 10.250.60.120: ICMP echo reply, id 11500, seq 2, length 64
And the test from client to server, with 2.3.10 succeeds:

Code: Select all

root@idspgw7-a:~# ping 10.250.60.120
PING 10.250.60.120 (10.250.60.120) 56(84) bytes of data.
64 bytes from 10.250.60.120: icmp_seq=1 ttl=64 time=0.813 ms
64 bytes from 10.250.60.120: icmp_seq=2 ttl=64 time=0.670 ms
tcpdump on the client:

Code: Select all

19:46:59.479822 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2020, seq 1, length 64
19:46:59.480576 IP 10.250.60.120 > 10.250.60.130: ICMP echo reply, id 2020, seq 1, length 64
19:47:00.480840 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2020, seq 2, length 64
19:47:00.481471 IP 10.250.60.120 > 10.250.60.130: ICMP echo reply, id 2020, seq 2, length 64
tcpdump on the server:

Code: Select all

19:46:59.523940 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2020, seq 1, length 64
19:46:59.524013 IP 10.250.60.120 > 10.250.60.130: ICMP echo reply, id 2020, seq 1, length 64
19:47:00.524931 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2020, seq 2, length 64
19:47:00.524976 IP 10.250.60.120 > 10.250.60.130: ICMP echo reply, id 2020, seq 2, length 64
And now repeat the tests with openerp-2.5_git from the master branch,
meaning I replace the binaries and re-run the experiment script on both sides,
same exact addressing, same exact routing rules and policies:

ping from client to the server bridge to prove routing:

Code: Select all

root@idspgw7-a:~# ping 10.250.120.120
PING 10.250.120.120 (10.250.120.120) 56(84) bytes of data.
64 bytes from 10.250.120.120: icmp_seq=1 ttl=64 time=0.574 ms
64 bytes from 10.250.120.120: icmp_seq=2 ttl=64 time=0.511 ms
tcpdump on client:

Code: Select all

19:55:19.197598 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2239, seq 1, length 64
19:55:19.198138 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2239, seq 1, length 64
19:55:20.197307 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2239, seq 2, length 64
19:55:20.197802 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2239, seq 2, length 64
tcpdump on server:

Code: Select all

19:55:19.232602 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2239, seq 1, length 64
19:55:19.232666 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2239, seq 1, length 64
19:55:20.232226 IP 10.250.130.130 > 10.250.120.120: ICMP echo request, id 2239, seq 2, length 64
19:55:20.232256 IP 10.250.120.120 > 10.250.130.130: ICMP echo reply, id 2239, seq 2, length 64
and ping client bridge from server to prove routing:

Code: Select all

[root@ahau1:~# ping 10.250.130.130
PING 10.250.130.130 (10.250.130.130) 56(84) bytes of data.
64 bytes from 10.250.130.130: icmp_seq=1 ttl=64 time=0.649 ms
64 bytes from 10.250.130.130: icmp_seq=2 ttl=64 time=0.375 ms
tcpdump on server:

Code: Select all

19:57:19.511575 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 30698, seq 1, length 64
19:57:19.512177 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 30698, seq 1, length 64
19:57:20.535237 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 30698, seq 2, length 64
19:57:20.535590 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 30698, seq 2, length 64
tcpdump on client:

Code: Select all

19:57:19.483475 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 30698, seq 1, length 64
19:57:19.483504 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 30698, seq 1, length 64
19:57:20.507030 IP 10.250.120.120 > 10.250.130.130: ICMP echo request, id 30698, seq 2, length 64
19:57:20.507048 IP 10.250.130.130 > 10.250.120.120: ICMP echo reply, id 30698, seq 2, length 64
And here is the testcase, from server to client, with openvpn 2.5_git

Code: Select all

root@ahau1:~# ping 10.250.60.130
PING 10.250.60.130 (10.250.60.130) 56(84) bytes of data.
^C
--- 10.250.60.130 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2044ms
tcpdump on server:

Code: Select all

19:59:12.730433 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 1072, seq 1, length 64
19:59:13.751246 IP 10.250.60.120 > 10.250.60.130: ICMP echo request, id 1072, seq 2, length 64
tcpdump on client:

Code: Select all

(empty)
And here is the testcase from client to server, openvpn-2.5_git:

Code: Select all

root@idspgw7-a:~# ping 10.250.60.120
PING 10.250.60.120 (10.250.60.120) 56(84) bytes of data.
^C
--- 10.250.60.120 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms


tcpdump on client:

Code: Select all

20:00:45.679662 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2241, seq 1, length 64
20:00:46.679376 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2241, seq 2, length 64
tcpdump on server

Code: Select all

(empty)
occasionally if restarting the server and client and re-running the test, I do see the a first reply
in the server tcpdump, never any indicator of seq 2 or seq 3:

Code: Select all

20:02:18.246623 IP 10.250.60.130 > 10.250.60.120: ICMP echo request, id 2370, seq 1, length 64
20:02:18.246704 IP 10.250.60.120 > 10.250.60.130: ICMP echo reply, id 2370, seq 1, length 64
(occasionally I see like that, stops here, reply never makes it back to client, seq 2 never arrives at server)
so the bottom line is, I wonder what is new in 2.5_git from 2.3.10,
and perhaps is there a setting or patch I could make so I can move my live servers forward
from the 2.3.10-ubuntu16 patch I applied over debian 9. I am happy with the patch for now....

forwarding config on server:

Code: Select all

root@ahau1:/etc/openvpn# for f in $(find /proc/sys -name rp_filter) ; do echo $f ; cat $f ; done
/proc/sys/net/ipv4/conf/all/rp_filter
0
/proc/sys/net/ipv4/conf/brserver/rp_filter
0
/proc/sys/net/ipv4/conf/default/rp_filter
0
/proc/sys/net/ipv4/conf/eth0/rp_filter
0
/proc/sys/net/ipv4/conf/eth1/rp_filter
0
/proc/sys/net/ipv4/conf/lo/rp_filter
0
/proc/sys/net/ipv4/conf/tuntestserver/rp_filter
0
root@ahau1:/etc/openvpn# cat /proc/sys/net/ipv4/ip_forward
1
forwarding config on client:

Code: Select all

root@idspgw7-a:/etc/openvpn# for f in $(find /proc/sys -name rp_filter) ; do echo $f ; cat $f ; done
/proc/sys/net/ipv4/conf/all/rp_filter
0
/proc/sys/net/ipv4/conf/brclient/rp_filter
0
/proc/sys/net/ipv4/conf/default/rp_filter
0
/proc/sys/net/ipv4/conf/enp0s25/rp_filter
0
/proc/sys/net/ipv4/conf/enp1s0/rp_filter
0
/proc/sys/net/ipv4/conf/enp2s0/rp_filter
0
/proc/sys/net/ipv4/conf/lo/rp_filter
0
/proc/sys/net/ipv4/conf/tuntestclient/rp_filter
0
root@idspgw7-a:/etc/openvpn# cat /proc/sys/net/ipv4/ip_forward
1
Gratitude.

TinCanTech
OpenVPN Protagonist
Posts: 11137
Joined: Fri Jun 03, 2016 1:17 pm

Re: sending packets whose destination address matches an IP on another interface on the peer

Post by TinCanTech » Thu Mar 22, 2018 6:07 pm

Because you have documented this so clearly (thank you), when I find some time, I will try to replicate this.

Post Reply