Page 1 of 1
Slow upload caused by iptables port redirection.
Posted: Sat Jul 10, 2021 1:05 pm
by vpnjumper
Hello,
I'm using OpenVPN 2.4.5 Server and currently experimenting with the following setup. Basically need to expose the same openvpn server on multiple ports:
Code: Select all
Client -> (uses "remote Server:443" in ovpn config)
Server:443 -> (iptables redirection to :1194)
Server:1194 (OpenVPN Server)
As iptables Rule I use this (tried also nginx udp reverse proxy, socat... same result)
Code: Select all
iptables -t nat -A PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
Connecting works fine, and download speed is unchanged. But when I run an upload speed test, speed is limited to about 4.5mbit/s. This does not happen if I connect to 1194 directly (then have 50mbit+). In server log I also get this when running upload speed test:
Code: Select all
PID_ERR large diff [601] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48260 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48260 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
PID_ERR large diff [600] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48261 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48261 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
This seems very strange, I also verified with tcpdump that the ovpn server gets all the packets. What is the reason for this?
Re: Slow upload caused by iptables port redirection.
Posted: Sat Jul 10, 2021 2:14 pm
by openvpn_inc
vpnjumper wrote: ↑Sat Jul 10, 2021 1:05 pm
I'm using OpenVPN 2.4.5 Server and currently experimenting with the following setup. Basically need to expose the same openvpn server on multiple ports:
Code: Select all
Client -> (uses "remote Server:443" in ovpn config)
Server:443 -> (iptables redirection to :1194)
Server:1194 (OpenVPN Server)
Why? Our
OpenVPN Access Server (proprietary software) product uses udp/1194 and tcp/443 by default (and in that order of preference, because OpenVPN performs best over UDP.) But I don't see much point in udp/443.
The reason for having tcp/443 is to get around certain "web-only" firewalls. Such firewalls are unlikely to allow any UDP ports.
vpnjumper wrote: ↑Sat Jul 10, 2021 1:05 pm
As iptables Rule I use this (tried also nginx udp reverse proxy, socat... same result)
Code: Select all
iptables -t nat -A PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
Connecting works fine, and download speed is unchanged. But when I run an upload speed test, speed is limited to about 4.5mbit/s. This does not happen if I connect to 1194 directly (then have 50mbit+). In server log I also get this when running upload speed test:
Code: Select all
PID_ERR large diff [601] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48260 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48260 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
PID_ERR large diff [600] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48261 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48261 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
This seems very strange, I also verified with tcpdump that the ovpn server gets all the packets. What is the reason for this?
Yes, it does seem strange. Best I can guess is that you're getting some replay packets, but I have no guess why udp/443 performance would differ from 1194. Show your work? How did you test this? Are you going through an asymmetric link? It's not uncommon for upload bandwidth to be a fraction of download.
Mystified, --rob0
P.S. I had this all written and then had an idea.

NAT rules generally need to be limited, yours is not. Your REDIRECT should only match packets arriving on the external interface. Supposing your external interface is
eth0, try:
Code: Select all
iptables -t nat -D PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
iptables -t nat -A PREROUTING -i eth0 -p udp --dport 443 -j REDIRECT --to-ports 1194
The first rule deletes your unrestricted NAT rule; the second adds one that's properly limited.
P.P.S. Don't use
iptables like this. Use
iptables-restore(8) to do a complete ruleset in one atomic operation. Your distro probably does this already.
Re: Slow upload caused by iptables port redirection.
Posted: Sat Jul 10, 2021 3:22 pm
by vpnjumper
openvpn_inc wrote: ↑Sat Jul 10, 2021 2:14 pm
Why? Our
OpenVPN Access Server (proprietary software) product uses udp/1194 and tcp/443 by default (and in that order of preference, because OpenVPN performs best over UDP.) But I don't see much point in udp/443.
QUIC or HTTP/3 seems to be using UDP/443. Allowing Port 443 Traffic regardless of protocol seems quite common in my experience.
Haven't looked at Access Server yet, thanks, might give it a try!
openvpn_inc wrote: ↑Sat Jul 10, 2021 2:14 pm
Yes, it does seem strange. Best I can guess is that you're getting some replay packets, but I have no guess why udp/443 performance would differ from 1194. Show your work? How did you test this? Are you going through an asymmetric link? It's not uncommon for upload bandwidth to be a fraction of download.
Nothing special, normal openvpn community server.
openvpn_inc wrote: ↑Sat Jul 10, 2021 2:14 pm
P.S. I had this all written and then had an idea.

NAT rules generally need to be limited, yours is not. Your REDIRECT should only match packets arriving on the external interface. Supposing your external interface is
eth0, try:
Code: Select all
iptables -t nat -D PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
iptables -t nat -A PREROUTING -i eth0 -p udp --dport 443 -j REDIRECT --to-ports 1194
The first rule deletes your unrestricted NAT rule; the second adds one that's properly limited.
Thanks, that solved it!
There was indeed an amplification happening, after resolving that upload speed is fine!
Re: Slow upload caused by iptables port redirection.
Posted: Sat Jul 10, 2021 4:27 pm
by openvpn_inc
Great! We learned something!

--rob0