VPN Tunnel with OpenVPN becomes slower and slower over time

This forum is for admins who are looking to build or expand their OpenVPN setup.
Forum rules
Please use the [oconf] BB tag for openvpn Configurations. See viewtopic.php?f=30&t=21589 for an example.
Post Reply
yoyo2020
OpenVpn Newbie
Posts: 2
Joined: Mon Nov 01, 2010 1:56 pm

VPN Tunnel with OpenVPN becomes slower and slower over time

Post by yoyo2020 » Mon Nov 01, 2010 2:05 pm

Hi Everyone,

We've got an OpenVPN setup (we control both server and clients) which is behaving somewhat oddly (otherwise, I won't post about it ... :) )

The server's public IP is 46.30.8.55 (I feel okay sharing this because it appears some script kiddies have already found our server and are port scanning it all the time). Pinging the server's public IP (not through a VPN tunnel) averages at about 100ms. Pinging the server's private IP, accessible via VPN, provides roughly 100ms when the tunnel is set up but over the course of a couple of hours it slows down considerably. Since it is not possible to attach a screenshot, I'll provide the details of the pings later on in this post (don't want to cause a mess so appending it at the end).

The server is a CentOS 5.3 and is always at 0-1% CPU (that is, almost 100% idle). The client in this case is Win7 64-bit. The logs of the connection from the OpenVPN client are also at the end of this post.

Restarting the OpenVPN client doesn't help. Restarting the computer appears to help. Even replacing a Netgear router with a Siemens one didn't help at all.

Any idea what may be causing this?

Thanks!
yi2020

PUBLIC IP PING:
Pinging 46.30.8.55 with 32 bytes of data:
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=114ms TTL=53
Reply from 46.30.8.55: bytes=32 time=114ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=108ms TTL=53
Reply from 46.30.8.55: bytes=32 time=102ms TTL=53
Reply from 46.30.8.55: bytes=32 time=108ms TTL=53
Reply from 46.30.8.55: bytes=32 time=112ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=222ms TTL=53
Reply from 46.30.8.55: bytes=32 time=111ms TTL=53
Reply from 46.30.8.55: bytes=32 time=106ms TTL=53
Reply from 46.30.8.55: bytes=32 time=103ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=95ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=111ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=109ms TTL=53
Reply from 46.30.8.55: bytes=32 time=104ms TTL=53
Reply from 46.30.8.55: bytes=32 time=103ms TTL=53
Reply from 46.30.8.55: bytes=32 time=266ms TTL=53
Reply from 46.30.8.55: bytes=32 time=333ms TTL=53
Reply from 46.30.8.55: bytes=32 time=199ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=150ms TTL=53
Reply from 46.30.8.55: bytes=32 time=134ms TTL=53
Reply from 46.30.8.55: bytes=32 time=95ms TTL=53
Reply from 46.30.8.55: bytes=32 time=138ms TTL=53
Reply from 46.30.8.55: bytes=32 time=95ms TTL=53
Reply from 46.30.8.55: bytes=32 time=109ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=95ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=103ms TTL=53
Reply from 46.30.8.55: bytes=32 time=101ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=96ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=106ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=106ms TTL=53
Reply from 46.30.8.55: bytes=32 time=99ms TTL=53
Reply from 46.30.8.55: bytes=32 time=98ms TTL=53
Reply from 46.30.8.55: bytes=32 time=101ms TTL=53
Reply from 46.30.8.55: bytes=32 time=100ms TTL=53
Reply from 46.30.8.55: bytes=32 time=97ms TTL=53
Reply from 46.30.8.55: bytes=32 time=104ms TTL=53

Ping statistics for 46.30.8.55:
Packets: Sent = 66, Received = 66, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 95ms, Maximum = 333ms, Average = 111ms

PRIVATE IP PING:
Pinging 10.3.1.75 with 32 bytes of data:
Reply from 10.3.1.75: bytes=32 time=594ms TTL=64
Reply from 10.3.1.75: bytes=32 time=546ms TTL=64
Reply from 10.3.1.75: bytes=32 time=348ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2458ms TTL=64
Reply from 10.3.1.75: bytes=32 time=325ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2492ms TTL=64
Reply from 10.3.1.75: bytes=32 time=770ms TTL=64
Reply from 10.3.1.75: bytes=32 time=961ms TTL=64
Reply from 10.3.1.75: bytes=32 time=279ms TTL=64
Reply from 10.3.1.75: bytes=32 time=497ms TTL=64
Reply from 10.3.1.75: bytes=32 time=388ms TTL=64
Reply from 10.3.1.75: bytes=32 time=304ms TTL=64
Reply from 10.3.1.75: bytes=32 time=759ms TTL=64
Reply from 10.3.1.75: bytes=32 time=646ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1713ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2412ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1523ms TTL=64
Reply from 10.3.1.75: bytes=32 time=674ms TTL=64
Reply from 10.3.1.75: bytes=32 time=826ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1629ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2008ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2546ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1319ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1427ms TTL=64
Reply from 10.3.1.75: bytes=32 time=693ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1088ms TTL=64
Reply from 10.3.1.75: bytes=32 time=226ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2131ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1064ms TTL=64
Reply from 10.3.1.75: bytes=32 time=748ms TTL=64
Reply from 10.3.1.75: bytes=32 time=525ms TTL=64
Reply from 10.3.1.75: bytes=32 time=2392ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1243ms TTL=64
Reply from 10.3.1.75: bytes=32 time=458ms TTL=64
Reply from 10.3.1.75: bytes=32 time=591ms TTL=64
Reply from 10.3.1.75: bytes=32 time=3296ms TTL=64
Reply from 10.3.1.75: bytes=32 time=1754ms TTL=64
Reply from 10.3.1.75: bytes=32 time=832ms TTL=64

Ping statistics for 10.3.1.75:
Packets: Sent = 38, Received = 38, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 226ms, Maximum = 3296ms, Average = 1170ms

OPENVPN CLIENT LOGS:
Mon Nov 01 13:37:27 2010 OpenVPN 2.1.3 i686-pc-mingw32 [SSL] [LZO2] [PKCS11] built on Aug 20 2010
Mon Nov 01 13:37:27 2010 WARNING: --ping should normally be used with --ping-restart or --ping-exit
Mon Nov 01 13:37:27 2010 NOTE: OpenVPN 2.1 requires '--script-security 2' or higher to call user-defined scripts or executables
Mon Nov 01 13:37:27 2010 LZO compression initialized
Mon Nov 01 13:37:27 2010 Control Channel MTU parms [ L:1544 D:140 EF:40 EB:0 ET:0 EL:0 ]
Mon Nov 01 13:37:27 2010 Socket Buffers: R=[8192->8192] S=[8192->8192]
Mon Nov 01 13:37:27 2010 Data Channel MTU parms [ L:1544 D:1450 EF:44 EB:135 ET:0 EL:0 AF:3/1 ]
Mon Nov 01 13:37:27 2010 Local Options hash (VER=V4): '69109d17'
Mon Nov 01 13:37:27 2010 Expected Remote Options hash (VER=V4): 'c0103fa8'
Mon Nov 01 13:37:27 2010 Attempting to establish TCP connection with 46.30.8.55:1723
Mon Nov 01 13:37:27 2010 TCP connection established with 46.30.8.55:1723
Mon Nov 01 13:37:27 2010 TCPv4_CLIENT link local: [undef]
Mon Nov 01 13:37:27 2010 TCPv4_CLIENT link remote: 46.30.8.55:1723
Mon Nov 01 13:37:27 2010 TLS: Initial packet from 46.30.8.55:1723, sid=80e7f777 f1a4bb31
Mon Nov 01 13:37:29 2010 VERIFY OK: depth=1, /C=IL/ST=IL/L=TelAviv/O=Fort/CN=Fort_CA/emailAddress=email@domain.com
Mon Nov 01 13:37:29 2010 VERIFY OK: nsCertType=SERVER
Mon Nov 01 13:37:29 2010 VERIFY OK: depth=0, /C=IL/ST=NA/L=TelAviv/O=Mycompany/OU=Indeni/CN=Mycompany.com/emailAddress=info@Mycompany.com
Mon Nov 01 13:37:32 2010 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 13:37:32 2010 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 13:37:32 2010 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 13:37:32 2010 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 13:37:32 2010 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA
Mon Nov 01 13:37:32 2010 [Mycompany.com] Peer Connection Initiated with 46.30.8.55:1723
Mon Nov 01 13:37:34 2010 SENT CONTROL [Mycompany.com]: 'PUSH_REQUEST' (status=1)
Mon Nov 01 13:37:34 2010 PUSH: Received control message: 'PUSH_REPLY,route 10.3.1.64 255.255.255.224,route 192.168.10.0 255.255.255.0,ping 10,ping-restart 120,ifconfig 192.168.10.6 192.168.10.5'
Mon Nov 01 13:37:34 2010 OPTIONS IMPORT: timers and/or timeouts modified
Mon Nov 01 13:37:34 2010 OPTIONS IMPORT: --ifconfig/up options modified
Mon Nov 01 13:37:34 2010 OPTIONS IMPORT: route options modified
Mon Nov 01 13:37:35 2010 ROUTE default_gateway=10.0.0.138
Mon Nov 01 13:37:35 2010 TAP-WIN32 device [Local Area Connection 3] opened: \\.\Global\{B7DFB62F-4C55-49CA-B3D3-75789B25A0B8}.tap
Mon Nov 01 13:37:35 2010 TAP-Win32 Driver Version 9.7
Mon Nov 01 13:37:35 2010 TAP-Win32 MTU=1500
Mon Nov 01 13:37:35 2010 Notified TAP-Win32 driver to set a DHCP IP/netmask of 192.168.10.6/255.255.255.252 on interface {B7DFB62F-4C55-49CA-B3D3-75789B25A0B8} [DHCP-serv: 192.168.10.5, lease-time: 31536000]
Mon Nov 01 13:37:35 2010 Successful ARP Flush on interface [20] {B7DFB62F-4C55-49CA-B3D3-75789B25A0B8}
Mon Nov 01 13:37:40 2010 TEST ROUTES: 2/2 succeeded len=2 ret=1 a=0 u/d=up
Mon Nov 01 13:37:40 2010 C:\WINDOWS\system32\route.exe ADD 10.3.1.64 MASK 255.255.255.224 192.168.10.5
Mon Nov 01 13:37:40 2010 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=30 and dwForwardType=4
Mon Nov 01 13:37:40 2010 Route addition via IPAPI succeeded [adaptive]
Mon Nov 01 13:37:40 2010 C:\WINDOWS\system32\route.exe ADD 192.168.10.0 MASK 255.255.255.0 192.168.10.5
Mon Nov 01 13:37:40 2010 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=30 and dwForwardType=4
Mon Nov 01 13:37:40 2010 Route addition via IPAPI succeeded [adaptive]
Mon Nov 01 13:37:40 2010 Initialization Sequence Completed
Mon Nov 01 14:37:32 2010 TLS: soft reset sec=0 bytes=7954555/0 pkts=21371/0
Mon Nov 01 14:37:34 2010 VERIFY OK: depth=1, /C=IL/ST=IL/L=TelAviv/O=Fort/CN=Fort_CA/emailAddress=email@domain.com
Mon Nov 01 14:37:34 2010 VERIFY OK: nsCertType=SERVER
Mon Nov 01 14:37:34 2010 VERIFY OK: depth=0, /C=IL/ST=NA/L=TelAviv/O=Mycompany/OU=Mycompany/CN=Mycompany.com/emailAddress=info@Mycompany.com
Mon Nov 01 14:37:37 2010 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 14:37:37 2010 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 14:37:37 2010 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 14:37:37 2010 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 14:37:37 2010 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA
Mon Nov 01 15:37:32 2010 TLS: tls_process: killed expiring key
Mon Nov 01 15:37:37 2010 TLS: soft reset sec=0 bytes=96222260/0 pkts=150383/0
Mon Nov 01 15:37:42 2010 VERIFY OK: depth=1, /C=IL/ST=IL/L=TelAviv/O=Fort/CN=Fort_CA/emailAddress=email@domain.com
Mon Nov 01 15:37:42 2010 VERIFY OK: nsCertType=SERVER
Mon Nov 01 15:37:42 2010 VERIFY OK: depth=0, /C=IL/ST=NA/L=TelAviv/O=Mycompany/OU=Mycompany/CN=Mycompany.com/emailAddress=info@Mycompany.com
Mon Nov 01 15:37:49 2010 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 15:37:49 2010 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 15:37:49 2010 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Nov 01 15:37:49 2010 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Nov 01 15:37:49 2010 Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 1024 bit RSA

User avatar
krzee
Forum Team
Posts: 729
Joined: Fri Aug 29, 2008 5:42 pm

Re: VPN Tunnel with OpenVPN becomes slower and slower over t

Post by krzee » Wed Nov 03, 2010 5:14 am

http://sites.inka.de/~bigred/devel/tcp-tcp.html

the weird part is that you say that restarting openvpn does not help...

have you tried --tcp-nodelay?

yoyo2020
OpenVpn Newbie
Posts: 2
Joined: Mon Nov 01, 2010 1:56 pm

Re: VPN Tunnel with OpenVPN becomes slower and slower over t

Post by yoyo2020 » Wed Nov 03, 2010 8:05 am

I forgot to mention that we were originally using UDP and thought it's possible TCP will be better. We're experiencing the same problems with both.

dazo
OpenVPN Inc.
Posts: 141
Joined: Mon Jan 11, 2010 10:14 am
Location: dazo :: #openvpn-devel @ irc.freenode.net

Re: VPN Tunnel with OpenVPN becomes slower and slower over t

Post by dazo » Wed Nov 03, 2010 8:27 am

I've just checked one of my sites. I got a site-to-site connection going between a Gentoo based OpenVPN server and an OpenVPN client on CentOS 5.5. Both sides are running OpenVPN 2.2-beta3. The connection has been up for 4 days without any issues. The ping time is also what I'd expect Meaning, the tunnel keeps the current speed.

As restarting the computer (the OS) helps and not when restarting OpenVPN, indicates a computer issue. That could be related to power saving or too little memory, where the OS spends time swapping applications between RAM and disk constantly. Unless there are some real oddities with the build on that particular platform. However, this latter point is less likely, unless you can get the same behaviour on another similar configured computer on the same OS.

Post Reply