Help! Pfctl with openvpn running on OSX w/ tunnelblick

kakky07

Hello,

First of all, I am new to the server admin side of things, so pardon my lack of knowledge.
I'm currently setting up a openvpn server on my macmini server at my house. My goal is to be able to connect to both my local network at my house as well as the internet through the vpn connection.
I found a thread on apple discussions that was doing something very similar with iOS here: https://discussions.apple.com/thread/4472353?start=0&tstart=0
which I followed for the most part.

It was working great for a while, but after setting up pfctl instead of ipfw, I can't seem to access anything on the local network or the internet when I attempt to connect from the outside. (The VPN still connects however).
I used icefloor http://www.hanynet.com/icefloor/ to setup pfctl.
On pfctl, I forwarded UDP port 1194 in the setup.

At this point, I'm not sure how I should troubleshoot this issue.
Any help would be great, thanks.

EDIT: I think the issue has to do with how I setup osx to route vpn packets.

I used the following script which I got from the OSX discussion:

#!/bin/bash
#
# Sleep is necessary cause network has to be up at the time of following commands
# Otherwise the network will not work at all
#
sleep 15
#
sysctl -w net.inet.ip.fw.enable=1
sysctl -w net.inet.ip.forwarding=1
natd -interface en0
ipfw add divert natd ip from any to any via en0

and I think its something that has to do with using

ipfw add divert natd ip from any to any via en0

How can I do the same thing with pfctl instead of ipfw?

stvs

Have you made any progress getting an OpenVPN server to work on OS X with pfctl? This appears to be necessary now on Mavericks 10.9, on which the above ipfw/natd commands no longer work.

stvs

I read The Boof of PF and figured it out. The essential pfctl NAT and filter rules are

nat on en0 from 10.0.0.0/8 to any -> (en0)
pass from { lo0, 10.0.0.0/8 } to any keep state

and you have to do a lot of trial-and-error to make sure you don't break your access while integrating these rules into OS X Server's basic rule set. A description of the complete setup with the following pf.conf is here.

sudo vi /etc/pf.conf

# References for modifications:
# The Book of PF by Peter N.M. Hansteen
# http://hints.macworld.com/article.php?story=20121011004626997
# http://blog.scottlowe.org/2013/05/15/using-pf-on-os-x-mountain-lion/
# http://krypted.com/mac-security/a-cheat-sheet-for-using-pf-in-os-x-lion-and-up/

# Options

set block-policy drop
set fingerprints "/etc/pf.os"
set ruleset-optimization basic

set skip on lo0

# Normalization

# Scrub incoming packets
scrub in all no-df

#
# com.apple anchor point
#
scrub-anchor "com.apple/*"

# Queueing

# Translation

# OpenVPN Server NAT
#
# The Book of PF, p. 21
int_if = "en0" # macro for internal interface
localnet = "10.0.0.0/8"
nat on $int_if from $localnet to any -> ($int_if)

nat-anchor "com.apple/*"
rdr-anchor "com.apple/*"
dummynet-anchor "com.apple/*"
anchor "com.apple/*"
load anchor "com.apple" from "/etc/pf.anchors/com.apple"

# Filtering

lan_server = 10.0.1.3

# Antispoof
antispoof log quick for { lo0 en0 }

# Block by default
block in log

# Allow outgoing traffic from NAT'd { lo0, $localnet }
# The Book of PF, p. 21
pass from { lo0, $localnet } to any keep state

# Block to/from illegal destinations or sources
block in log quick from no-route to any

# Allow critical system traffic
pass in quick inet proto udp from any port 67 to any port 68

# Allow ICMP from home LAN
pass in log proto icmp from $lan_server:network

# Allow outgoing traffic
pass out inet proto tcp from any to any keep state
pass out inet proto udp from any to any keep state

# Internet services
internet_udp_services = "{ https, 500, 1194, 1701, 4500, 5060, 5190, 5297, 5298, 5678, 16384 }"
internet_tcp_services = "{ ssh, smtp, https, 143, 587, 993, 995, 1640, 2170, 2195, 2196, 4190,\
5218, 5223, 5190, 5220, 5222, 5298, 8008, 8443, 8800, 8843 }"
pass in quick inet proto tcp from any to { lo0, $lan_server } port $internet_tcp_services
pass in quick inet proto udp from any to { lo0, $lan_server } port $internet_udp_services

# LAN services: block access, except from localnet
lan_udp_services = "{ 5001 }"
lan_tcp_services = "{ domain, auth, nntp, www, 311, 3128, 5001, 5900:5909, 8118, 8123 }"
pass in quick inet proto tcp from { lo0, $localnet } to { lo0, $lan_server } port $lan_tcp_services
pass in quick inet proto udp from { lo0, $localnet } to { lo0, $lan_server } port $lan_udp_services

stvs

Still some issues getting packets between VPN clients and services on the VPN Server. Any pointers would be appreciated. IceFloor is a really nice GUI, but also has this issue, so this approach is missing some basic ingredient.

For example, hitting a VNC server while running tcpdump shows VNC packets going to the server on interface tun0, and leaving the VNC server on interface en0. I'm not sure if packets are correctly crossing over between tun0 and en0 -- the result is that services are not available to VPN clients.

More details for those who'd be able to suggest a fix:

tcpdump shows VNC packets going to the server on interface tun0, and leaving the VNC server on interface en0. I'm not sure if packets are correctly crossing over between tun0 and en0 -- the result is that services are not available to VPN clients.

Is anyone able to suggest pfctl or other fixes that allow VPN clients on tun0 to access services on the same VPN server available on interface en0?

Here's the dump of tun0 showing packets going to the VNC service on tun0. The dump above shows them leaving the VNC service on interface en0. Are they supposed to be routed back to tun0 somehow? How?

I can ping my server from VPN clients, but nmap fails, and VNC clients fail to hit 10.0.1.3:5900. Yet tcpdump shows traffic on port 5900 associated with the VPN client. All this stuff works fine when not on VPN and sitting on the LAN.

ping from VPN:

iPad$ ping -c 3 10.0.1.3
PING 10.0.1.3 (10.0.1.3): 56 data bytes
64 bytes from 10.0.1.3: icmp_seq=0 ttl=64 time=38.303 ms
64 bytes from 10.0.1.3: icmp_seq=1 ttl=64 time=37.200 ms
64 bytes from 10.0.1.3: icmp_seq=2 ttl=64 time=36.507 ms
--- 10.0.1.98 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 36.507/37.337/38.303/0.740 ms

nmap from VPN client:

iPad$ nmap -p 5900 10.0.1.3
Starting Nmap 5.00 ( http://nmap.org ) at 2013-11-10 11:00 EST
Note: Host seems down. If it is really up, but blocking our ping probes , try -PN
Nmap done: 1 IP address (0 hosts up) scanned in 3.28 seconds

tcpdump on server interfaces en0 and tun0 during VPN client access:

server$ sudo tcpdump -n -e -ttt -i en0 port 5900
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
00:00:00.000000 c4:2f:03:0b:01:9d > b8:37:5d:de:dc:24, ethertype IPv4 (0x0800), length 78: 10.0.1.3.5900 > 10.8.0.10.52892: Flags [S.], seq 500616894, ack 2287587651, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 922913452 ecr 546523456,sackOK,eol], length 0
00:00:05.881495 c4:2f:03:0b:01:9d > b8:37:5d:de:dc:24, ethertype IPv4 (0x0800), length 78: 10.0.1.3.5900 > 10.8.0.10.52892: Flags [S.], seq 500616894, ack 2287587651, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 922918265 ecr 546523456,sackOK,eol], length 0 00:00:04.620650 c4:2c:03:0b:51:9d > b8:c7:5d:d0:dc:14, ethertype

server$ sudo tcpdump -n -e -ttt -i tun0 port 5900

00:00:00.142140 AF IPv4 (2), length 68: 10.8.0.10.53295 > 10.0.1.3.5900: Flags [S], seq 2223347580, win 65535, options [mss 1368,nop,wscale 4,nop,nop,TS val 566298513 ecr 0,sackOK,eol], length 0