Hello,
I have main router, ASUS RT-AC3200 running Merlin FW connected to WAN.
RT-AC3200
IP 192.168.10.0
Subnet 255.255.0.0
Gateway 192.168.10.1
I've setup two OpenVPN servers (OpenVPN 2.4.4, Community edition) on a laptop running PfSense. Laptop is connected to LAN of RT-AC3200. NAT is disabled. Remote clients use OpenVPN v. 2.4.4-1 and are all Linux VMs.
Pfsense
IP 192.168.20.1
Subnet 255.255.255.0
Server #1 TUN
IP 10.20.30.0
Subnet 255.255.255.0
Server #2 TAP
IP 192.168.20.0
Subnet 255.255.255.0
Gateway 192.168.20.1
dev ovpns1
verb 4
dev-type tap
dev-node /dev/tap1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
cipher AES-128-OFB
auth SHA512
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local 192.168.20.1
engine cryptodev
tls-server
mode server
username-as-common-name
auth-user-pass-verify "xxxx" via-env
tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'abc.com' 1"
lport 3000
management /var/etc/openvpn/server1.sock unix
max-clients 30
push "dhcp-option DNS 192.168.10.1"
push "redirect-gateway def1"
client-to-client
duplicate-cn
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /etc/dh-parameters.4096
tls-auth /var/etc/openvpn/server1.tls-auth 0
ncp-ciphers AES-128-OFB:AES-192-OFB:AES-256-OFB
comp-lzo no
push "comp-lzo no"
persist-remote-ip
float
auth-nocache
push "route-gateway 192.168.10.1"
sndbuf 393216
rcvbuf 393216
push "sndbuf 393216"
push "rcvbuf 393216"
fragment 1348
mssfix 1308
dev tap
persist-tun
persist-key
#cipher AES-256-CBC
cipher AES-128-OFB
ncp-ciphers AES-128-OFB:AES-192-OFB:AES-256-OFB
auth SHA512
tls-client
client
resolv-retry infinite
remote abc.com 2002 udp
lport 0
verify-x509-name "abc.com" name
auth-user-pass auth.txt
remote-cert-tls server
auth-nocache
verb 5
#mtu-test
fragment 1348
mssfix 1308
#mtu-disc yes
compress
ifconfig 192.168.10.40 255.255.0.0
The purpose of this setup is to connect
Server #2 TAP clients to the LAN of RT-AC3200, so all clients are accessible on both sides. The use of
Server #1 TUN is to be able to reach LAN clients of RT-AC3200 and
Server #1 TUN clients with iOS and Android devices.
I've successfully setup both servers. The
Server #1 TUN works perfect with great speeds up and down. But I am experiencing an unstable connection and, what it seems, packet loss on
Server #2 TAP. The transfer speed is terrible when fetching files and it goes up and down like a yo-yo on SMB (100kB/s - 1500kB/s). I also experience jitter lags when accessing clients in web browsers and SSH. Both local network and remote network have 100/100Mbit/s from ISP, hence it should not be the bottle neck.
An example from Wireshark output which occurs on a high rate when e.g. transferring larger files:
Source Destination Protocol Length Info
Local LAN Client Remote TAP client TCP 66 [TCP Dup ACK 64713#1] 21663 → 445 [ACK] Seq=14393 Ack=43518308 Win=243456 Len=0 SLE=43519431 SRE=43520554
I started investigating on MTU and I found out when pinging from remote TAP client to RT-AC3200 gateway that MTU is not standard 1500B. The highest value that was successful was 1348B:
$ ping -I tap0 -c5 -M do -s 1348 192.168.10.1
PING 192.168.10.1 (192.168.10.1) from 192.168.10.55 tap0: 1348(1376) bytes of data.
1356 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=57.2 ms
1356 bytes from 192.168.10.1: icmp_seq=2 ttl=64 time=27.4 ms
1356 bytes from 192.168.10.1: icmp_seq=3 ttl=64 time=27.6 ms
1356 bytes from 192.168.10.1: icmp_seq=4 ttl=64 time=27.1 ms
1356 bytes from 192.168.10.1: icmp_seq=5 ttl=64 time=28.0 ms
I've been trying to solve this by adding
fragment 1348 and
mssfix 1308 and the speed went up to 300kB/s - 2.5Mbit/s, but still acting as a yo-yo and I still see same output in Wireshark as shown above.
I did the mtu-test:
NOTE: Empirical MTU test completed [Tried,Actual] local->remote=[1617,1617] remote->local=[1614,1566]
This connection is unable to accommodate a UDP packet size of 1617. Consider using --fragment or --mssfix options as a workaround.
I'm not sure how to proceed. All help is appreciated.
VPN client log:
do_ifconfig, tt->did_ifconfig_ipv6_setup=0
/sbin/ip link set dev tap0 up mtu 1500
/sbin/ip addr add dev tap0 192.168.10.50/16 broadcast 192.168.255.255
/sbin/ip route add 195.198.189.113/32 via 158.34.54.1
/sbin/ip route add 0.0.0.0/1 via 192.168.10.1
/sbin/ip route add 192.0.0.0/1 via 192.168.10.1
Initialization Sequence Completed
Note that all IP addresses are fake.
