Tweaking pf for OpenVPN on Mavericks server (Mac OS X 10.9)

jlg89

I'm running both the "stock" VPN service and an OpenVPN server on Mac OS X 10.9.5 Server. The server has two physical network interfaces, one with public routed IPs and the other with private IPs. The public interface has four different public IPs, all of which are working properly. The "stock" VPN (PPTP/L2TP) is configured to assign addresses in the 10.0.77.0/24 range. My OpenVPN interface, utun0, uses a 10.0.88.0/24 range, and is configured to only listen on one of the public IPs.

I've added a pf rule to create a NAT behind the primary public IP for any non-public interface:
nat on en0 from !(en0) to any -> (en0:0)

With that rule in effect, NAT works perfectly for the private interface and the PPTP/L2TP VPN clients. OpenVPN clients can access the internet, and can access the server's public IP on which the OpenVPN server is listening; however, they can't access the server's other public IPs. I've tried various pf rules to try to fix this, but it has me completely stymied. I'm new to pf, though, so I have to think it's probably something simple.

Any ideas would be most appreciated.

innogen

jlg89 wrote
Any ideas would be most appreciated.

Friend, I don't think this is the right sub-forum to post questions on networking.

Perhaps you might like to google for an appropriate forum, Stackexchange site or Reddit.

jlg89

Apologies if I'm in the wrong place (I've been wrong before), but I assumed that "Server Administration" would cover issues related to administrating an OpenVPN server. There are specific mods that must be made on Mavericks to get pf to work properly with OpenVPN, and nobody in any of the myriad of forums I've searched has come up with a working solution. I'm thinking this forum would be a Really Good Place(tm) to post that solution, once we find it.

debbie10t

jlg89 wroteApologies if I'm in the wrong place

If your post was deemed inappropriate it would have been disapproved in the first place, as it has been approved, it can remain.

On the face of it your issue appears to be routing/NAT .. Posting complete details of your setup may help others to understand.

This forum will very likely only provide support for OpenVPN.

So, if you care to remove your other PPTP & L2TP vpns from the equation it may be easier to advise.

jlg89 wrotecan access the server's public IP on which the OpenVPN server is listening; however, they can't access the server's other public IPs

Does your server allow loop back connections to your other public IPs .. have you also checked your firewall ?

jlg89

It might not be a pf issue at all, but perhaps a routing issue? The PPTP/L2TP VPN range is 10.0.77/24, the OpenVPN range is 10.0.88/24. The server has four public IPs, let's call them 5.5.5.1, 5.5.5.2, 5.5.5.3, and 5.5.5.4. If the OpenVPN server is listening on 5.5.5.1, then OpenVPN clients can hit the public internet, and can hit other services listening on 5.5.5.1. But they can't hit any services listening on 5.5.5.2-5.5.5.4. PPTP/L2TP clients have no problems.

Looking at the routing table, I notice that each PPTP/L2TP VPN client gets its own route dynamically added (e.g., 10.0.77.33), but that doesn't happen for OpenVPN clients:

Destination Gateway Flags Refs Use Netif Expire
10.0.77/24 link#5 UCS 1 0 en1
10.0.77.1 localhost UHS 0 0 lo0
10.0.77.33 link#5 UHLWI 0 4 en1
10.0.88/24 10.0.88.2 UGSc 2 108 utun0
10.0.88.2 10.0.88.1 UH 3 0 utun0

I also notice that I can't ping anything on the 10.0.88/24 network from the server. I tried "route add 10.0.88.1 localhost" which fixed the ping issue but not the OpenVPN client routing issue.

jlg89

Further data.

An OpenVPN client _can_ ping the other public interfaces, either by IP address or by DNS name, but cannot connect to services listening on those addresses (e.g. web, mail, etc.). Running a traceroute to the other public IPs from an OpenVPN client returns a single hop to the selected public IP...which is odd, since traceroutes to other internet hosts begin with the OpenVPN tunnel gateway (10.0.88.1).

jlg89

Here's the server's openvpn.conf file, with the public IP obfuscated.

local 5.5.5.1
port 1194
proto udp
dev tun
ca cert/ca.crt
cert cert/server.crt
key cert/server.key
crl-verify cert/crl.pem
dh cert/dh2048.pem
server 10.0.88.0 255.255.255.0
push "redirect-gateway"
push "dhcp-option DNS 10.0.88.1"
keepalive 10 120
tls-auth cert/ta.key 0
comp-lzo
max-clients 30
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
verb 4
mute 10

jlg89

If I connect an OpenVPN client (to the server listening on 5.5.5.1), then try to hit http://5.5.5.2 from the client, I see this in the output of "pfctl -s state" (the web page never loads):

ALL tcp 5.5.5.2:80 <- 10.0.88.6:53458 SYN_SENT:ESTABLISHED
ALL tcp 5.5.5.2:80 <- 10.0.88.6:53460 SYN_SENT:ESTABLISHED

The status stays like that for a bit, then changes to TIME_WAIT:TIME_WAIT, then finally times out. Nothing shows up in the apache access log for 5.5.5.2:80, so apparently the request isn't making it from the OpenVPN tunnel to the public IP.

If I connect an L2TP VPN client, to the same server interface, and hit the same URL, I get this (the web page loads):

ALL tcp 5.5.5.2:80 <- 10.0.77.35:53491 ESTABLISHED:ESTABLISHED

I include this info because the pf rules, as far as I can tell, are identical for the two private networks, in fact nothing is blocked for the private networks at all. And again, general internet access works just fine for the OpenVPN clients, they're just unable to access services on the server's other public IPs. So the NAT function is working, and no outgoing traffic is being blocked, but something is preventing traffic flow between the other public IPs and the OpenVPN tunnel.

innogen

To: jlg89

Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

http://serverfault.com/

Network Engineering Stack Exchange is a question and answer site for network engineers. It's 100% free, no registration required.

http://networkengineering.stackexchange.com/

jlg89

OK...best as I can tell, this is not a pf issue after all. I stripped the pf ruleset down to the NAT declaration and a "pass all" and the OpenVPN client issue remains. It has to be some kind of quirky routing issue.

So...what would cause OpenVPN clients to be able to access everything except other public IPs on the OpenVPN server itself?

Traffic

Try NAT-ing the client traffic on the server ... should be easy enough on MAC OSX.

Something like:

iptables -t nat A POSTROOTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

jlg89

NAT is in place (Mavericks uses pf instead of iptables) and working--OpenVPN and raccoon (L2TP/PPTP) clients can access the internet. The raccoon clients can also access services on the server's other IP addresses, but OpenVPN clients can't.

OpenVPN clients can ping and traceroute to the server's other IPs, so ICMP traffic is getting through, but nothing else. Something is preventing OpenVPN traffic from being able to loop back to the server's IPs.

I've stripped the pf ruleset down to:

set skip on { lo0 utun0 utun1 utun2 }
nat on en0 from !(en0) to any -> (en0:0)
pass all flags S/SA keep state

OpenVPN clients still can't get anything but ICMP to the server's IPs.

Traffic

How about if you ping the OpenVPN client from the source address of the public IPs which apparently cannot see the VPN ?
Not just the machine but set the source address to be the public IP ..

jlg89

Good call. No, pings sent from any of the public IPs to an OpenVPN client private IP get no response. "Standard" pings from the server succeed.

Traffic

OK .. sounds like routing then ..

as your setup is quite complicated I am not going to pretend I know what routing you need but I would say the public IP interface needs appropriate routing for your vpn.

jlg89

If I disable the "nat on en0..." pf rule, all VPN clients can access all server IPs. But when I enable the nat rule, OpenVPN clients lose that ability. They can still access the internet, just nothing on the server itself. The racoon VPN clients don't have this issue. There has to be some kind of pf rule tweak that I'm missing.

jlg89

Turns out I just needed a "rdr" rule right after the nat declaration, to redirect anything coming from an OpenVPN client destined for the server's public IP to the OpenVPN virtual gateway:

nat on en0 from ! (en0) to any -> (en0:0)
rdr pass on utun0 inet proto { tcp udp } from 10.0.88.0/24 to en0 -> 10.0.88.1

Apparently racoon does this on its own (?) but OpenVPN does not. I still can't figure why pings work without the rule but not tcp/udp--but that's why I only redirect tcp/udp with the rule. This pf one-liner was MUCH easier than screwing with split-horizon DNS.