[RESOLVED] Inacceptable openVPN performance

gridrun

Hi all,

I'm trying to use openVPN to secure a WLAN. The setup looks like this:

[Client PC]---802.11n---[AP] ---100mbps Ethernet---[Firewall]--100mbps Ethernet--[CPE]---[Internet]

Client PC is a Windows 7 netbook (Atom N455 @ 1.66GHz, 2GB RAM), the Firewall is a virtualized pfSense 2.1 box (E1000 NICs, 2 vCPUs, 512MB RAM) running on vSphere 5 (core 2 duo @ 2.4GHz, 8GB RAM). The internet connection is a 100mbps down/7mbps up DOCSIS 3.0 cable pipe.

Testing speeds without openVPN, the setup consistently delivers >80mbps download (using both speedtest.net and cnlabs speedtest) and the full 7mbps upload right behind the Firewall (not going through the WLAN). When measuring at the client PC (though WLAN), download speed is around 33mbps without openVPN (which seems about right, considering that its really just an 802.11n "lite" WLAN).

With openVPN in udp mode, download speed drops to 4.7mbps max. I tried about a gazillion of different tun-mtu/fragment/mssfix combinations, which only made openVPN perform even worse.

I tried with and without LZO compression. I tried with and without encryption. No matter what, 4.7mbps max. Notably, it doesn't make any difference if I disable encryption (cipher none).

Monitoring the CPU load on both pfSense/vSphere and Client box revealed nothing unusual. I also tried using a different client computer with way more CPU power (core 2 quad, 2.66GHz, 16GB RAM), but to no avail.

What makes a difference is using TCP instead: 8.89mbps down/1.67mbps up. But that's as good as it gets. Disabling openVPN, we're back to ~ 33mbps down/7mbps up (over the WLAN).

I'm pulling hairs here. I have no idea what else to try.

Anyone have a suggestion for me?

janjust

where is the openvpn server in this picture? what is the raw speedtest performance of the openvpn server itself.
As an alternative to OpenVPN , try SSH'ing into the OpenVPN server with dynamic forwarding ('-D 1080') and then run speedtest on your local PC using a SOCKS5 server @ 127.0.0.1:1080

gridrun

Hey,

The openVPN server is on the [Firewall]. It is a pfSense 2.1 box.
RAW speed through the Firewall (without using openVPN) is as expected (>80mbps down).

Weird, huh?

janjust

run 'iperf' from the openvpn client to the openvpn server (using their VPN IPs) - what kind of performance does that give?

(iperf is available for Linux, Windows and MacOS; alternatively you can simply upload or download a large file from the openvpn client to the server.)

gridrun

Hi Jan,

I've done testing with iperf and here are the results.

First, I was testing plain WLAN speeds from [Windows 7 Client] to [Firewall] over WLAN, without openVPN.

Test Run A - Without openVPN
-------------------------------------

Test #1 client command line: iperf -c <firewall WLAN IP>
results: Interval=0.0-10.0sec, Transfer=29.8MBytes, Bandwidth=24.9Mbits/s

Test #2 client command line: iperf -l 32768 -c <firewall WLAN IP>
results: Interval=0.0-10.0sec, Transfer=39.8MBytes, Bandwidth=33.3Mbits/s

Test #3 client command line: iperf -w 32768 -l 65535 -c <firewall WLAN IP>
results: Interval=0.0-10.0sec, Transfer=36.6MBytes, Bandwith=30.7Mbits/s

Second, I checked the speed over the VPN. Note that the openVPN server is on the Firewall box (pfSense)!

Test Run B - With openVPN
----------------------------------

Test #1 client command line: iperf -c <OpenVPN Server IP>
Results: Interval=0.0-10.4sec, Transfer=944KBytes, Bandwidth=743Kbits/s

Test #2 client command line: iperf -l 32768 -c <openVPN Server IP>
Results: Interval=0.0-10.5sec, Transfer=1.88MBytes, Bandwidth=1.50Mbits/s

Test #3 client command line: iperf -w 32768 -l 65535 -c <openVPN Server IP>
Results: Interval=0.0-10.9sec, Transfer=1.94MBytes, Bandwidth=1.50Mbits/s

From these tests, OpenVPN over the WLAN appears to be 22 times slower than the WLAN itself.

gridrun

I just found out that setting

net.inet.ip.fastforwarding = 1

on the server provides an almost 90% performance boost :O

janjust

cool!
so performance is now close to optimal?
I didn't know about this flag, but then found out it is pfSense/BSD specific :-(

gridrun

Well,

Definitely a giant leap in getting there! I have just applied the aforementioned setting (it is fbsd specific, yes) to the WLAN infrastructure and performed some more iperf testing to validate the results. Here they are:

Environment is the same network and involved machines as in runs A and B. All test runs C and D are however done with ip.fastforwarding = 1. No other changes have been made to the openVPN. Compare the results to the previous tests in my earlier post:

Test Run C - Without OpenVPN
------------------------------

Test #1 client command line: iperf -c <firewall WLAN IP>
results: Interval=0.0-10.0sec, Transfer=31.4MBytes, Bandwidth=26.3Mbits/sec

Test#2 client command line: iperf -l 32768 -c <firewall WLAN IP>
results: Interval=0.0-10.0sec, Transfer=39.5MBytes, Bandwidth=33.1Mbits/sec

Test#3: omitted.

Test Run D - With OpenVPN
---------------------------

Test #1 client command line: iperf -c <OpenVPN Server IP>
results: Interval=0.0-10.0sec, Transfer=16.0MBytes, Bandwidth=13.4Mbits/sec

Test #2 client command line: iperf -l 32768 -c <OpenVPN Server IP>
results: Interval=0.0-10.0sec, Transfer=17.1MBytes, Bandwidth=14.0Mbits/sec

Test #3: omitted.

Compare Test Runs B and D. Conclusion: What a HUGE freakin' difference :!: :!: :!:
Finally, the bandwidth limit is by traffic shaping (14600kbps) and not longer by openVPN.

For those interested, here's the corresponding thread on pfsense.org forum:
http://forum.pfsense.org/index.php/topic,47567.0.html

gridrun

After almost one month with ip.fastforwarding=1 on our pfSense firewalls, I (and my users) have not found any adverse effects.

Still keeping an eye. Will let you know if we have some news.