Slow upload caused by iptables port redirection.

This forum is for admins who are looking to build or expand their OpenVPN setup.

Moderators: TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech

Forum rules
Please use the [oconf] BB tag for openvpn Configurations. See viewtopic.php?f=30&t=21589 for an example.
Post Reply
vpnjumper
OpenVpn Newbie
Posts: 2
Joined: Sat Jul 10, 2021 12:53 pm

Slow upload caused by iptables port redirection.

Post by vpnjumper » Sat Jul 10, 2021 1:05 pm

Hello,

I'm using OpenVPN 2.4.5 Server and currently experimenting with the following setup. Basically need to expose the same openvpn server on multiple ports:

Code: Select all

Client -> (uses "remote Server:443" in ovpn config)
Server:443 -> (iptables redirection to :1194)
Server:1194 (OpenVPN Server)
As iptables Rule I use this (tried also nginx udp reverse proxy, socat... same result)

Code: Select all

iptables -t nat -A PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
Connecting works fine, and download speed is unchanged. But when I run an upload speed test, speed is limited to about 4.5mbit/s. This does not happen if I connect to 1194 directly (then have 50mbit+). In server log I also get this when running upload speed test:

Code: Select all

PID_ERR large diff [601] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48260 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48260 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
PID_ERR large diff [600] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48261 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48261 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
This seems very strange, I also verified with tcpdump that the ovpn server gets all the packets. What is the reason for this?

User avatar
openvpn_inc
OpenVPN Inc.
Posts: 1333
Joined: Tue Feb 16, 2021 10:41 am

Re: Slow upload caused by iptables port redirection.

Post by openvpn_inc » Sat Jul 10, 2021 2:14 pm

vpnjumper wrote:
Sat Jul 10, 2021 1:05 pm
I'm using OpenVPN 2.4.5 Server and currently experimenting with the following setup. Basically need to expose the same openvpn server on multiple ports:

Code: Select all

Client -> (uses "remote Server:443" in ovpn config)
Server:443 -> (iptables redirection to :1194)
Server:1194 (OpenVPN Server)
Why? Our OpenVPN Access Server (proprietary software) product uses udp/1194 and tcp/443 by default (and in that order of preference, because OpenVPN performs best over UDP.) But I don't see much point in udp/443.

The reason for having tcp/443 is to get around certain "web-only" firewalls. Such firewalls are unlikely to allow any UDP ports.
vpnjumper wrote:
Sat Jul 10, 2021 1:05 pm
As iptables Rule I use this (tried also nginx udp reverse proxy, socat... same result)

Code: Select all

iptables -t nat -A PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
Connecting works fine, and download speed is unchanged. But when I run an upload speed test, speed is limited to about 4.5mbit/s. This does not happen if I connect to 1194 directly (then have 50mbit+). In server log I also get this when running upload speed test:

Code: Select all

PID_ERR large diff [601] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48260 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48260 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
PID_ERR large diff [600] [SSL-0] [0000000000000000000000000000000000000000000000000000000000000000] 0:48861 0:48261 t=1625683565[0] r=[-4,64,15,982,1] sl=[41,64,64,528]
AEAD Decrypt error: bad packet ID (may be a replay): [ #48261 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
This seems very strange, I also verified with tcpdump that the ovpn server gets all the packets. What is the reason for this?
Yes, it does seem strange. Best I can guess is that you're getting some replay packets, but I have no guess why udp/443 performance would differ from 1194. Show your work? How did you test this? Are you going through an asymmetric link? It's not uncommon for upload bandwidth to be a fraction of download.

Mystified, --rob0

P.S. I had this all written and then had an idea. :) NAT rules generally need to be limited, yours is not. Your REDIRECT should only match packets arriving on the external interface. Supposing your external interface is eth0, try:

Code: Select all

iptables -t nat -D PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
iptables -t nat -A PREROUTING -i eth0 -p udp --dport 443 -j REDIRECT --to-ports 1194
The first rule deletes your unrestricted NAT rule; the second adds one that's properly limited.

P.P.S. Don't use iptables like this. Use iptables-restore(8) to do a complete ruleset in one atomic operation. Your distro probably does this already.
Image OpenVPN Inc.
Answers provided by OpenVPN Inc. staff members here are provided on a voluntary best-effort basis, and no rights can be claimed on the basis of answers posted in this public forum. If you wish to get official support from OpenVPN Inc. please use the official support ticket system: https://openvpn.net/support

vpnjumper
OpenVpn Newbie
Posts: 2
Joined: Sat Jul 10, 2021 12:53 pm

Re: Slow upload caused by iptables port redirection.

Post by vpnjumper » Sat Jul 10, 2021 3:22 pm

openvpn_inc wrote:
Sat Jul 10, 2021 2:14 pm
Why? Our OpenVPN Access Server (proprietary software) product uses udp/1194 and tcp/443 by default (and in that order of preference, because OpenVPN performs best over UDP.) But I don't see much point in udp/443.
QUIC or HTTP/3 seems to be using UDP/443. Allowing Port 443 Traffic regardless of protocol seems quite common in my experience.
Haven't looked at Access Server yet, thanks, might give it a try!
openvpn_inc wrote:
Sat Jul 10, 2021 2:14 pm
Yes, it does seem strange. Best I can guess is that you're getting some replay packets, but I have no guess why udp/443 performance would differ from 1194. Show your work? How did you test this? Are you going through an asymmetric link? It's not uncommon for upload bandwidth to be a fraction of download.
Nothing special, normal openvpn community server.
openvpn_inc wrote:
Sat Jul 10, 2021 2:14 pm
P.S. I had this all written and then had an idea. :) NAT rules generally need to be limited, yours is not. Your REDIRECT should only match packets arriving on the external interface. Supposing your external interface is eth0, try:

Code: Select all

iptables -t nat -D PREROUTING -p udp --dport 443 -j REDIRECT --to-ports 1194
iptables -t nat -A PREROUTING -i eth0 -p udp --dport 443 -j REDIRECT --to-ports 1194
The first rule deletes your unrestricted NAT rule; the second adds one that's properly limited.
Thanks, that solved it! :D :D
There was indeed an amplification happening, after resolving that upload speed is fine!

User avatar
openvpn_inc
OpenVPN Inc.
Posts: 1333
Joined: Tue Feb 16, 2021 10:41 am

Re: Slow upload caused by iptables port redirection.

Post by openvpn_inc » Sat Jul 10, 2021 4:27 pm

Great! We learned something! :) --rob0
Image OpenVPN Inc.
Answers provided by OpenVPN Inc. staff members here are provided on a voluntary best-effort basis, and no rights can be claimed on the basis of answers posted in this public forum. If you wish to get official support from OpenVPN Inc. please use the official support ticket system: https://openvpn.net/support

Post Reply