I'm currently really confused ... and No, this is not a normal state for me ;-) My problem: I'd a setup, working since month which let me connect to networks so I can reach each machine from other network. Now, end of last week it was necessary to replace to current client hardware (device, called pogoplug but modified with a debian installation) with a newer one (zyxel NAS 540). So far, the NAS currently owns his own firmware and modding with debian or other distribution is currently not possible because kernel is compiled with 64k page size which breaks compatibility which many available binarys for ARM arch. But some community users made openvpn available with the needed compile settings.
Network setup
Network A
On network A i've running LXC (vmm-h01). I created a container (app-h01) inside which is running the openVPN Server (IP: 192.168.200.204). The setup between host and containers are bridged with veth interfaces, container IPs are in the same network like others. It's necessary because inside one container I've a service broadcast/multicast content and therefor it must be in the same network.
Network A is a 192.168.200.0/24 network.
On router (192.168.200.254) following rules are set:
Network Subnetmask Gateway
192.168.201.0 255.255.255.0 192.168.200.204
192.168.254.0 255.255.255.0 192.168.200.204
Network B
On network B the openVPN client (IP: 192.168.201.200) is running directly on the NAS without any containers or virtualization.
Network A is a 192.168.201.0/24 network.
On router (192.168.201.254) following rules are set:
Network Subnetmask Gateway
192.168.200.0 255.255.255.0 192.168.201.200
192.168.254.0 255.255.255.0 192.168.201.200
The problem, confuse me
I install openvpn on the nas, copy the old setup to it and starts openvpn. Connection was directly established and all looks fine. I ping from network A a host on network B, works! Pings from network B a host on network A, works! Next I want ssh to the host on the network B on which the openVPN client is running ... fail! I can ping the host but I can't ssh to it. So I ssh to a host inside network on network B and ssh from there to the host with openVPN client and it works ... and the behaviour is the same (=reversed) when I try it from network B to A ...
Analyzing problem
To analyze, whats wrong, i tracepath from a host inside network A:
... to a host inside network B
leonard:~ # tracepath 192.168.201.254
1?: [LOCALHOST] pmtu 8996
1: vmm-h01.fritz.box 0.776ms pmtu 1500
1: app-v01.fritz.box 0.577ms
2: 192.168.254.2 22.156ms
3: 192.168.201.254 26.285ms reached
Resume: pmtu 1500 hops 3 back 62
... to the host with openVPN Client on network B
leonard:~ # tracepath 192.168.201.200
1?: [LOCALHOST] pmtu 8996
1: vmm-h01.fritz.box 0.753ms pmtu 1500
1: app-v01.fritz.box 0.594ms
2: 192.168.201.200 20.341ms reached
Resume: pmtu 1500 hops 2 back 63
Now, the way is fine up to host app-v01.fritz.box. Here is a difference: On first tracepath the packages "hop" via 192.168.254.2, on the second it directly hop to 192.168.201.200. I've currenlty no further idea, whats wrong. I would understand, nothing would work, i would understand if i could reach only services directly on the openVPN instances (because of wrong or missing routing) but I don't understand, why all works excepted reaching services on the openVPN instances.
I also try it with a openVPN directly on vmm-h01 excluding strange routing/briding behavours because of LXC setup but the result is the same ...
OpenVPN Server version
OpenVPN 2.3.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Dec 1 2014
library versions: OpenSSL 1.0.1k 8 Jan 2015, LZO 2.08
Originally developed by James Yonan
Copyright (C) 2002-2010 OpenVPN Technologies, Inc. <sales@openvpn.net>
Compile time defines: enable_crypto=yes enable_debug=yes enable_def_auth=yes enable_dependency_tracking=no enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_fast_install=yes enable_fragment=yes enable_http_proxy=yes enable_iproute2=yes enable_libtool_lock=yes enable_lzo=yes enable_lzo_stub=no enable_maintainer_mode=no enable_management=yes enable_multi=yes enable_multihome=yes enable_pam_dlopen=no enable_password_save=yes enable_pedantic=no enable_pf=yes enable_pkcs11=yes enable_plugin_auth_pam=yes enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_server=yes enable_shared=yes enable_shared_with_static_runtimes=no enable_small=no enable_socks=yes enable_ssl=yes enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=yes enable_win32_dll=yes enable_x509_alt_username=yes with_crypto_library=openssl with_gnu_ld=yes with_ifconfig_path=/sbin/ifconfig with_iproute_path=/sbin/ip with_mem_check=no with_plugindir='${prefix}/lib/openvpn' with_route_path=/sbin/route with_sysroot=no
git revision: refs/heads/jessie/b35ad09bfc4a26e7
server.confdaemon
port 8070
dev tun
proto tcp-server
ifconfig 192.168.254.1 192.168.254.2
secret /etc/openvpn/static.key
script-security 2
up /etc/openvpn/server.up
down /etc/openvpn/server.down
status /var/log/openvpn-status.log
log-append /var/log/openvpn.log
comp-lzo
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
user nobody
group nogroup
verb 3
server.up#!/bin/bash
sysctl -w net/ipv4/conf/eth0/proxy_arp=1
sysctl -w net/ipv4/ip_forward=1
route add -net 192.168.254.0/24 dev tun0
route add -net 192.168.201.0/24 dev tun0
exit 0
server.down#!/bin/bash
route del -net 192.168.201.0/24 dev tun0
route del -net 192.168.254.0/24 dev tun0
exit 0
Routing table OpenVPN ServerKernel-IP-Routentabelle
Ziel Router Genmask Flags Metric Ref Use Iface
default 192.168.200.254 0.0.0.0 UG 0 0 0 eth0
192.168.200.0 * 255.255.255.0 U 0 0 0 eth0
192.168.201.0 * 255.255.255.0 U 0 0 0 tun0
192.168.254.0 * 255.255.255.0 U 0 0 0 tun0
192.168.254.2 * 255.255.255.255 UH 0 0 0 tun0
----------
OpenVPN Client VersionOpenVPN 2.3.6 armv7l-unknown-linux-gnueabi [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6] built on Apr 1 2015
library versions: OpenSSL 1.0.1j 15 Oct 2014, LZO 2.09
Originally developed by James Yonan
Copyright (C) 2002-2010 OpenVPN Technologies, Inc. <sales@openvpn.net>
Compile time defines: enable_crypto='yes' enable_crypto_ofb_cfb='yes' enable_debug='yes' enable_def_auth='yes' enable_dlopen='unknown' enable_dlopen_self='unknown' enable_dlopen_self_static='unknown' enable_fast_install='yes' enable_fragment='yes' enable_http_proxy='yes' enable_iproute2='no' enable_libtool_lock='yes' enable_lzo='yes' enable_lzo_stub='no' enable_management='yes' enable_multi='yes' enable_multihome='yes' enable_pam_dlopen='no' enable_password_save='no' enable_pedantic='no' enable_pf='yes' enable_pkcs11='no' enable_plugin_auth_pam='yes' enable_plugin_down_root='yes' enable_plugins='yes' enable_port_share='yes' enable_selinux='no' enable_server='yes' enable_shared='yes' enable_shared_with_static_runtimes='no' enable_small='no' enable_socks='yes' enable_ssl='yes' enable_static='yes' enable_strict='no' enable_strict_options='no' enable_systemd='no' enable_win32_dll='yes' enable_x509_alt_username='no' with_crypto_library='openssl' with_gnu_ld='yes' with_mem_check='no' with_plugindir='$(libdir)/openvpn/plugins' with_sysroot='no'
client.conf# Set remote address
remote <...>
dev tun
proto tcp-client
ifconfig 192.168.254.2 192.168.254.1
secret /etc/openvpn/static.key
script-security 2
up /etc/openvpn/client.up
down /etc/openvpn/client.down
status /var/log/openvpn-status.log
log-append /var/log/openvpn.log
comp-lzo
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
user nobody
group nogroup
verb 2
client.up#!/opt/bin/bash
openvpn --mktun --dev tun0
echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp
echo 1 > /proc/sys/net/ipv4/ip_forward
route add -net 192.168.200.0/24 gw 192.168.254.2 dev tun0
exit 0
client.down#!/opt/bin/bash
route del -net 192.168.200.0/24 dev tun0
exit 0
Routing table OpenVPN ClientDestination Gateway Genmask Flags Metric Ref Use Iface
default fritz.box 0.0.0.0 UG 5 0 0 egiga0
192.168.200.0 192.168.254.2 255.255.255.0 UG 0 0 0 tun0
192.168.201.0 * 255.255.255.0 U 0 0 0 egiga0
192.168.254.1 * 255.255.255.255 UH 0 0 0 tun0
