Hello.
I have recently faced this very issue and in extensive research found bits and pieces of information around this subject but not the total and complete solution.
Therefore I'd like to share the solution to the problem.
Note this guide is based around linux, I'm not sure how to do the same in windows, but I imagine it is possible, perhaps some 3rd party software allows to configure it.
The issue:
Routing is traditionally done on level 3 of OSi reference model. This implies that "traditional" routing is based around hosts or networks. So i.e. it is possible to say, all traffick from/to this host/network go via this gateway/interface.
The challenge:
We want to route traffick based on source or destination port, which is something that "traditional" Level 3 routing does not support.
I my particular case, I'm using commercial "anonymising" VPN service. OpenVPN config is provided by the service, and once openvpn client is started then all the traffick is redirected via vpn. However then http server no longer works (which is using the public, nonvpn ip). This machine is also serving as Wireless access point, so it routes packets between wlan and internet.
The solution:
Linux iproute2 has the capability of routing packets based on iptables rules, so anything you can specify as an iptable rule can be routed differently.
Very important:
ipforwarding has to be enabled and rp_filter has to be disabled for involved interfaces.
Especially the later is hardly mentioned in any documentation, and these days it is enabled in most distros. Having this enabled will most ikely screw up your attempts to mark and route the packets. the rp_filter, aka Reverse Path Filter, it inspects incomming packet and checks whether the the packet has valid return path (via gateway). If not it will drop the packet. I was tearing my hair out, seeing incomming http requests to the server in the tcpdump, however apache would not log any access attempts. This is because rp_filter was dropping them. It would not show in iptables counters.
Anyways, without further ado:
to check for ip_forwarding and rp_filter:
Code:
sysctl net.ipv4.conf.all.rp_filter = 1
sysctl -a | grep \\.rp_filter
sysctl net.ipv4.conf.[interface].rp_filter = 0
Then create a new routing table:
Code:
echo 100 openvpn >> /etc/iproute2/rt_tables
ip route add default via <nonvpn gateway ip> dev <nonvpn interface> table nonvpn
ip route show table nonvpn
Add ip forwarding rule
Code:
ip rule mark fwmark 1 lookup nonvpn
ip rule show
Create iptables rules to mark packets intended to route via nonvpn table. Those rules need to go to the 'mangle' iptable. Rules which mark packets originating from the localhost have to go into the 'OUTPUT' chain, whereas rules marking packets from different networks go to 'PREROUTING' chain
Code:
iptables -A OUTPUT -t mangle -p tcp --sport 80 -j MARK --set-mark=1 //reply packets from http server
iptables -A OUTPUT -t mangle -p tcp --sport 22 -j MARK --set-mark=1 //reply packets from ssh
iptables -A PREROUTING -t mangle -i wlan0 -j MARK --set-mark=1 //packets from wlan on dev box
That's all.
Troubleshooting tools i found usefull:
1. Use tcpdump or wireshark to see the packets sent/recieved on the relevant interface(s)
2. reset iptables counters (-Z) and then generate packets and see the counters as to which tables/chains/rules packets get to and which not.
Credits:
http://www.tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.netfilter.htmlhttp://lartc.org/howto/lartc.cookbook.fullnat.intro.htmlhttp://www.wlug.org.nz/SourceBasedRoutinghttp://www.tolaris.com/2009/07/13/disabling-reverse-path-filtering-in-complex-networks/
