OpenVPN+Shorewall. Internal routing OK, external fails

gymmarn

I'm trying to setup OpenVPN. I've got the VPN to work and the routing for the internal network 192.168.30.0/24 seems OK. But I can't seem to get it to route the VPN clients so they can access Internet through the routers external interface 77.66.55.44.

Summary:
* OpenVPN OK
* Internat routing OK
* External routing NOK (need help)

I'm really hoping someone can help me with this.

Here's my setup on the server:

network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth1
iface eth1 inet static
        address 77.66.55.44
        netmask 255.255.255.0
        gateway 77.66.55.1

auto eth2
iface eth2 inet static
        address 192.168.30.200
        netmask 255.255.255.0

openvpn/server.conf

local 77.66.55.44
proto tcp
port 1194
dev tun
server 192.168.31.0 255.255.255.0
push "route 192.168.30.0 255.255.255.0"
push "dhcp-option DNS 192.168.31.1"
push "redirect-gateway def1"
ifconfig-pool-persist ipp.txt
client-to-client
comp-lzo
keepalive 10 60
ping-timer-rem
persist-tun
persist-key

#Server keys
ca ca.crt
cert server.crt
key server.key
dh dh1024.pem
tls-auth ta.key 0
cipher BF-CBC

shorewall/interfaces

net     eth1            detect          dhcp,routefilter,tcpflags,logmartians,nosmurfs
loc     eth2            detect
vpn    tun+            -          routeback

shorewall/masq

#INTERFACE              SOURCE          ADDRESS         PROTO   PORT(S) IPSEC   MARK
eth1                    eth2
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

shorewall/policy

#SOURCE         DEST            POLICY          LOG LEVEL       LIMIT:BURST
$FW             all             ACCEPT  $LOG
vpn             all             ACCEPT  $LOG
loc             all             ACCEPT  $LOG
net             all             REJECT    $LOG
# The FOLLOWING POLICY MUST BE LAST
all             all             REJECT          $LOG
#LAST LINE -- ADD YOUR ENTRIES ABOVE THIS LINE -- DO NOT REMOVE

shorewall/rules

#OpenVPN
ACCEPT:$LOG             net                             $FW             tcp     1194
ACCEPT:$LOG             net                             $FW             udp     1194
#

shorewall/tunnels

openvpnserver:udp:1194      net     0.0.0.0/0
#LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE

shorewall/zones

fw      firewall
net     ipv4
loc     ipv4
vpn     ipv4
#LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONE - DO NOT REMOVE

shorewall/shorewall.conf

STARTUP_ENABLED=Yes
VERBOSITY=1
SHOREWALL_COMPILER=
LOGFILE=/var/log/shorewall.log
STARTUP_LOG=
LOG_VERBOSITY=
LOGFORMAT="Shorewall:%s:%s:"
LOGTAGONLY=No
LOGRATE=
LOGBURST=
LOGALLNEW=
BLACKLIST_LOGLEVEL=$LOG
MACLIST_LOG_LEVEL=$LOG
TCP_FLAGS_LOG_LEVEL=$LOG
SMURF_LOG_LEVEL=$LOG
LOG_MARTIANS=Yes
IPTABLES=
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
SHOREWALL_SHELL=/bin/sh
SUBSYSLOCK=""
MODULESDIR=
CONFIG_PATH=/etc/shorewall:/usr/share/shorewall
RESTOREFILE=
IPSECFILE=zones
LOCKFILE=
DROP_DEFAULT="Drop"
REJECT_DEFAULT="Reject"
ACCEPT_DEFAULT="none"
QUEUE_DEFAULT="none"
NFQUEUE_DEFAULT="none"
RSH_COMMAND='ssh ${root}@${system} ${command}'
RCP_COMMAND='scp ${files} ${root}@${system}:${destination}'
IP_FORWARDING=Yes
ADD_IP_ALIASES=Yes
ADD_SNAT_ALIASES=No
RETAIN_ALIASES=No
TC_ENABLED=Internal
TC_EXPERT=No
CLEAR_TC=Yes
MARK_IN_FORWARD_CHAIN=No
CLAMPMSS=No
ROUTE_FILTER=Yes
DETECT_DNAT_IPADDRS=No
MUTEX_TIMEOUT=60
ADMINISABSENTMINDED=Yes
BLACKLISTNEWONLY=Yes
DELAYBLACKLISTLOAD=No
MODULE_SUFFIX=
DISABLE_IPV6=Yes
BRIDGING=No
DYNAMIC_ZONES=No
PKTTYPE=Yes
RFC1918_STRICT=No
MACLIST_TABLE=filter
MACLIST_TTL=
SAVE_IPSETS=No
MAPOLDACTIONS=No
FASTACCEPT=No
IMPLICIT_CONTINUE=No
HIGH_ROUTE_MARKS=No
USE_ACTIONS=Yes
OPTIMIZE=0
EXPORTPARAMS=Yes
EXPAND_POLICIES=Yes
KEEP_RT_TABLES=No
DELETE_THEN_ADD=Yes
MULTICAST=No
DONT_LOAD=
AUTO_COMMENT=Yes
MANGLE_ENABLED=Yes
USE_DEFAULT_RT=No
RESTORE_DEFAULT_ROUTE=Yes
FAST_STOP=No
BLACKLIST_DISPOSITION=DROP
MACLIST_DISPOSITION=REJECT
TCP_FLAGS_DISPOSITION=DROP

And now to my client config (Windows computers running OpenVPN Client 2.2.3)

client
dev tun
remote 77.66.55.44 1194
resolv-retry infinite
proto udp
nobind
persist-key
persist-tun
ca ca.crt
cert client.crt
key client.key
tls-auth ta.key 1
cipher BF-CBC
comp-lzo
verb 3
mute 500
route-method exe
route-delay 5
ns-cert-type server
reneg-sec 0

gymmarn

Update: When I try the command openvpn --config /etc/openvpn/tun0.conf --verb 6 I get some weird stuff at the end.

Sat Dec 31 10:35:52 2011 us=322942 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Sat Dec 31 10:35:52 2011 us=322993 /sbin/ifconfig tun0 192.168.31.1 pointopoint 192.168.31.2 mtu 1500
Sat Dec 31 10:35:52 2011 us=325733 /sbin/route add -net 192.168.31.0 netmask 255.255.255.0 gw 192.168.31.2
Sat Dec 31 10:35:52 2011 us=327098 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:135 ET:0 EL:0 AF:3/1 ]
Sat Dec 31 10:35:52 2011 us=327161 UDPv4 link local (bound): [AF_INET]77.66.55.44:1194
Sat Dec 31 10:35:52 2011 us=327182 UDPv4 link remote: [undef]
Sat Dec 31 10:35:52 2011 us=327213 MULTI: multi_init called, r=256 v=256
Sat Dec 31 10:35:52 2011 us=327285 IFCONFIG POOL: base=192.168.31.4 size=62, ipv6=0
Sat Dec 31 10:35:52 2011 us=327336 ifconfig_pool_read(), in='gunnar,192.168.31.4', TODO: IPv6
Sat Dec 31 10:35:52 2011 us=327365 succeeded -> ifconfig_pool_set()
Sat Dec 31 10:35:52 2011 us=327385 IFCONFIG POOL LIST
Sat Dec 31 10:35:52 2011 us=327405 gunnar,192.168.31.4
Sat Dec 31 10:35:52 2011 us=327458 Initialization Sequence Completed
^CSat Dec 31 10:37:37 2011 us=461642 event_wait : Interrupted system call (code=4)
Sat Dec 31 10:37:37 2011 us=461848 TCP/UDP: Closing socket
Sat Dec 31 10:37:37 2011 us=461911 /sbin/route del -net 192.168.31.0 netmask 255.255.255.0
Sat Dec 31 10:37:37 2011 us=463346 Closing TUN/TAP interface
Sat Dec 31 10:37:37 2011 us=463394 /sbin/ifconfig tun0 0.0.0.0
Sat Dec 31 10:37:37 2011 us=564167 SIGINT[hard,] received, process exiting

It seems it adds a route for /sbin/route add -net 192.168.31.0 netmask 255.255.255.0 gw 192.168.31.2 but my tun0 interface is 192.168.31.1

gymmarn

Trying to manually set the route didn't help my external routing. Internal worked though :/

I've also checked my shorewall log, it seems to allow the packets. So I think the problem is that my router doesn't know how to route back packets that goes to eth1 and then needs to be sent back to tun0.

Dec 31 10:43:53 fw kernel: [ 1226.836166] Shorewall:vpn2net:ACCEPT:IN=tun0 OUT=eth1 SRC=192.168.31.6 DST=144.63.250.10 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=1484 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1591
Dec 31 10:43:58 fw kernel: [ 1231.836061] Shorewall:vpn2net:ACCEPT:IN=tun0 OUT=eth1 SRC=192.168.31.6 DST=144.63.250.10 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=1490 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1597
Dec 31 10:44:08 fw kernel: [ 1241.318777] Shorewall:fw2net:ACCEPT:IN= OUT=eth1 SRC=77.66.55.44 DST=89.233.250.255 LEN=254 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=234
Dec 31 10:44:08 fw kernel: [ 1241.324542] Shorewall:fw2net:ACCEPT:IN= OUT=eth1 SRC=77.66.55.44 DST=89.233.250.255 LEN=234 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=214
Dec 31 10:44:08 fw kernel: [ 1241.331950] Shorewall:fw2loc:ACCEPT:IN= OUT=eth2 SRC=192.168.30.200 DST=192.168.30.255 LEN=254 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=234
Dec 31 10:44:08 fw kernel: [ 1241.339630] Shorewall:loc2fw:ACCEPT:IN=eth2 OUT= MAC= SRC=192.168.30.200 DST=192.168.30.255 LEN=254 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=234
Dec 31 10:44:08 fw kernel: [ 1241.347459] Shorewall:fw2loc:ACCEPT:IN= OUT=eth2 SRC=192.168.30.200 DST=192.168.30.255 LEN=234 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=214
Dec 31 10:44:08 fw kernel: [ 1241.355198] Shorewall:loc2fw:ACCEPT:IN=eth2 OUT= MAC= SRC=192.168.30.200 DST=192.168.30.255 LEN=234 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=UDP SPT=138 DPT=138 LEN=214
Dec 31 10:44:08 fw kernel: [ 1241.836056] Shorewall:vpn2net:ACCEPT:IN=tun0 OUT=eth1 SRC=192.168.31.6 DST=144.63.250.10 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=1502 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1609
Dec 31 10:44:12 fw kernel: [ 1245.057785] Shorewall:net2fw:ACCEPT:IN=eth1 OUT= MAC=00:19:99:3e:3e:fa:00:23:34:42:99:c2:08:00 SRC=85.8.31.212 DST=77.66.55.44 LEN=65 TOS=0x00 PREC=0x00 TTL=58 ID=0 DF PROTO=UDP SPT=47887 DPT=53 LEN=45
Dec 31 10:44:13 fw kernel: [ 1246.836233] Shorewall:vpn2net:ACCEPT:IN=tun0 OUT=eth1 SRC=192.168.31.6 DST=144.63.250.10 LEN=60 TOS=0x00 PREC=0x00 TTL=127 ID=1508 PROTO=ICMP TYPE=8 CODE=0 ID=1 SEQ=1615

gymmarn

Oh, btw, I'm open to throwing Shorewall out. But then I need some help with setting up iptables rules which I'm not so familiar with (only know how to open ports). I need the Ubuntu server to route traffic från eth2 to eth1 so that the computers behind it can get to the Internet.

gymmarn

For anyone else with my problems. I solved it by throwing out Shorewall and adding:

iptables -t nat -I POSTROUTING -o eth1 -j MASQUERADE
iptables -A INPUT -i tun+ -j ACCEPT
iptables -A FORWARD -i tun+ -o eth1 -j ACCEPT
iptables -A FORWARD -i eth1 -o tun+ -j ACCEPT
iptables -A INPUT -p tcp --dport 1194 -j ACCEPT
iptables -A INPUT -p udp --dport 1194 -j ACCEPT
iptables -A FORWARD -i tun+ -o eth2 -j ACCEPT
iptables -A FORWARD -i eth2 -o tun+ -j ACCEPT

dazo

gymmarn wrote iptables -A FORWARD -i tun+ -o eth1 -j ACCEPT
iptables -A FORWARD -i eth1 -o tun+ -j ACCEPT

Be very careful with these lines. They completely disables firewalling between all tun devices and eth1 (which is the ethernet device with "internet", if I've read things correctly). This is a very insecure firewall configuration.

A more proper way would be:

iptables -A FORWARD -i eth1 -o tun+ -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i tun+ -o eth1 -m conntrack --ctstate NEW,ESTABLISHED,RELATED -j ACCEPT

This will allow new connections to be established from the VPN side to the outside world, and the packets going back from an already established connection back to the VPN clients. However new connections from the outside will not be allowed.