SSH freezes over OpenVPN when output > 3 - 5 lines

[avenn]

Hi,

I have been working with open VPN networks for a couple of years now and not really had any real issues.

However in the last few weeks. I'm having strange SSH over open VPN problems. Simplistically when I SSH from the server into an open VPN client. If I run a command that outputs 3+ lines such as ps aux the terminal freezes.

For example, when trying to identify open VPN versions running on the client's I get the 1st 3 lines and then it freezes:


OpenVPN 2.2.1 arm-linux-gnueabihf [SSL] [LZO2] [EPOLL] [PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on Apr 28 2012
Originally developed by James Yonan
Copyright (C) 2002-2010 OpenVPN Technologies, Inc. <sales@openvpn.net>[/color]


After researching the issue. It appears as though it is fairly recent. See the website links below:

http://askubuntu.com/questions/239146/o ... e-to-12-10


I have tried:

The solution at http://wandin.net/dotclear/index.php?po ... N-MTU-Size

When I run mtu–test local and remote MTU size match. Running #fragment 1400
#--mssfix make the issue worse - no connection.

Server specification:
Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-48-generic x86_64)
OpenVPN 2.2.1 x86_64-linux-gnu [SSL] [LZO2] [EPOLL] [PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on Feb 27 2013

Client Specification
2013-02-09-wheezy-raspbian - Debian-based raspberry pi image
OpenVPN 2.2.1 arm-linux-gnueabihf [SSL] [LZO2] [EPOLL] [PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on Apr 28 2012

I can un freeze the terminal by creating another SSH connection and killing the process. I can ping to the server VPN tunnel through out, which doesn't appear to have disconnected.

On the frozen terminal - which is now unfrozen it outputs: Write failed: Broken pipe

I looked at upgrading of the clients open VPN is as they appear to be an earlier build. However, as the clients upgrade process outputs more than 3 to 5 lines I am unable to do so remotely.

The network does not appear to slow down as in when sending pings to the clients that freeze replies continue with almost identical response times.

I recall when the clients were in the office and I SSH to them here they were no issues in relation to outputting numerous lines of information - for example top

it does appear as though this occurs when I run SSH over open VPN.

I recently ran an upgrade on the server and I wonder whether or not this is when the issue occurred?

Can anybody please help. Thank you.

Regards

Aidan

Server configurations :

dev tun0
proto udp

server 10.8.0.0 255.255.0.0

client-config-dir ccd

#A connecting client has client config file in client config dir
;ccd-exclusive

#revoke certificate list below GOTO http://openvpn.net/index.php/open-sourc ... howto.html
;crl-verify /etc/openvpn/crl.pem

#mtu-test
#fragment 1400
#--mssfix

tls-server
# Diffie-Hellman Parameters (tls-server only)
dh /etc/openvpn/easy-rsa/2.0/keys/dh1024.pem
# Certificate Authority file
ca /etc/openvpn/easy-rsa/2.0/keys/ca.crt
# Our certificate/public key
cert /etc/openvpn/easy-rsa/2.0/keys/server.crt
# Our private key
key /etc/openvpn/easy-rsa/2.0/keys/server.key

user nobody
group nogroup

#persist-tun
persist-key

#PUSH SETTINGS TO CLIENTS
push "ping 15"
push "ping-restart 90"
push "persist-tun"
push "persist-key"

# 3 -- medium output, good for normal operation.
# 9 -- verbose, good for troubleshooting

verb 3
log-append server.log

Client configuration:

#client4

dev tun0
proto udp
remote *********.com
pull

mtu-test

remote-cert-tls server

tls-client
# Certificate Authority file
ca /etc/openvpn/easy-rsa/keys/ca.crt
# Our certificate/public key
cert /etc/openvpn/easy-rsa/keys/client4.crt
# Our private key
key /etc/openvpn/easy-rsa/keys/client4.key

user nobody
group nogroup

# 3 -- medium output, good for normal operation.
# 9 -- verbose, good for troubleshooting

verb 3
log-append client4.log

Client ccd file:

#client4
ifconfig-push 10.8.0.17 10.8.0.18

[maikcat]

other services f.e ftp,smb,http work ok?

of you expirience issues when massive traffic is generated
then its usually mtu issue..

btw when ssh "freezes" can yuo still ping the client or the entire vpn channel drops?

if you use wireshark can you check if there something unusual?

Michael.

[avenn]

other services f.e ftp,smb,http work ok?

***sftp appears to stop working

***ftp appears ok (client download from server):

ftp 10.8.0.1
Connected to 10.8.0.1.
220 (vsFTPd 2.3.5)
Name (10.8.0.1:zabbix): zabbix
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> get zabbix-2.0.5.tar.gz
local: zabbix-2.0.5.tar.gz remote: zabbix-2.0.5.tar.gz
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for zabbix-2.0.5.tar.gz (13352744 bytes).
226 Transfer complete.
13352744 bytes received in 12.03 secs (1084.3 kB/s)
ftp>

of you expirience issues when massive traffic is generated
then its usually mtu issue..

btw when ssh "freezes" can yuo still ping the client or the entire vpn channel drops?

***yes i can still ping the client - delay still the same as prior to no ssh connection:

64 bytes from 10.8.0.13: icmp_req=127 ttl=64 time=33.4 ms


if you use wireshark can you check if there something unusual?

I`m going to look at upgrading openvpn on the clients - will have to go to site. Will wireshark the network.

I noticed that when the CLI output was slow as in processing an apt-get install it worked fine - it appears to be when the output is very fast like ps aux or top

Any suggestions?

Regards

Aidan

[avenn]

Ran nuttcp and got:

sudo nuttcp 10.8.0.1
nuttcp-t: v6.1.2: Error: server not ACKing data

intersting?

[maikcat]

zabbix?

are there ANY ip table rules on server/client? (rate limit... :S )

Michael.