Poor performance in AWS VPC

mcacker

We have developed an OpenVPN mesh between several AWS VPC regions, and although the mesh is functional, we are very concerned about the performance.

Using iperf to test the bandwidth between the OpenVPN servers and clients we get the following results:

with OpenVPN off, and testing bandwith directly between 2 servers over the public IP, we get around 100Mbits/sec.


/usr/local/bin/iperf3 -s -p 1224
/usr/local/bin/iperf3 -c 54.x.x.x  -p 1224
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   115 MBytes  96.7 Mbits/sec    1             sender
[  4]   0.00-10.00  sec   114 MBytes  95.7 Mbits/sec                  receiver

with OpenVPN we get approx 5MBits/sec for UDP
with OpenVPN we get approx 2.5MBits/sec for TCP.

/usr/local/bin/iperf3 -s -p 1224
/usr/local/bin/iperf3 -c 10.x.x.x  -p 1224
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  7.05 MBytes  5.91 Mbits/sec   34             sender
[  4]   0.00-10.00  sec  6.93 MBytes  5.81 Mbits/sec                  receiver

Is this the best that we can expect?

We are using Amazon Linux (amzn-ami-pv-2014.03.0.x86_64-ebs), and OpenVPN 2.3.2 or 2.3.4 (similar results for either)

I did take a look at https://forums.openvpn.net/post41228.html?hilit=throughput#p41228, and hence tested 2.3.4, which made no difference.

Examples of our configuration files below.

thanks, Mitchell

Server:

dev tun6
proto udp
port 1200
ifconfig 10.0.0.14 10.0.0.13
topology p2p
verb 4
secret /etc/openvpn/key
ping 15
ping-restart 300
resolv-retry 300
ping-timer-rem
persist-tun
persist-key
user nobody
group openvpn
status /var/log/openvpn-status.log
log /var/log/openvpn.log
route 10.y.y.y 255.255.255.0

Client:

remote 54.x.x.x
dev tun6
proto udp
port 1200
ifconfig 10.0.0.13 10.0.0.14
topology p2p
verb 4
secret /etc/openvpn/key
ping 15
ping-restart 300
resolv-retry 300
ping-timer-rem
persist-tun
persist-key
user nobody
group openvpn
status /var/log/openvpn-status.log
log /var/log/openvpn.log
route 10.x.x.x 255.255.255.0

mcacker

I should add that top shows openvpn at ~ 3% CPU, and there is no other traffic on the tunnel (or any tunnels on that server)

mcacker

The mesh is simple a matrix of tunnels from each region to every other region, or to be more precise, from each availability zone to each other availability zone. Each tunnel is on a separate tunnel number, i.e., tunN, and port.

For the purposes of this discussion, we can consider a tunnel in isolation as I've shut all the other tunnels down between the servers in question.

mcacker

thanks for taking the time to look at it.

FYI, I just re-ran the tests on Ubuntu 14.04 (the same as your published AMI), with HVM virtualization, a larger instance type, and got better results ~10 MBits/sec, but still nowhere near the capacity of the connection (100 MBits/sec), or what we would hope to see.