LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Business solution to host your own OpenVPN server with web management interface and bundled clients.
Post Reply
User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Fri Jul 09, 2021 12:04 am

**ENVIRONMENT:**

I have successfully set up a LEMP email/webserver running off of my home IP address, with my domains dns pointing to my home IP. My server is behind a firewall router, which is behind my ISP router. I believe this would be called a double NAT setup. To get all of the email/web server ports working, I had to forward port 443, port 80, and the corresponding email server ports, from my 'ISP router' to my 'firewall router', then, forward those same ports from my 'firewall router' to my 'LEMP Webserver'.

**THE PROBLEM:**

My issues is that my home IP is a not a static IP, so I cannot set the proper Reverse DNS and PTR records for the email server. In an attempt to solve this, I have created a cloud AWS Openvpn Access Server EC2 Instance, and have assigned it an elastic IP, which now has the proper PTR and reverse DNS records for my server.

**THE GOAL:**

The idea, was to tunnel the webserver as a client to the AWS EC2 Openvpn Access Server instance for the sole purpose of using and AWS IP address, DNS address, and setting the proper PTR Records on amazons elastic IP's settings. The desired outcome for all of this, would be that my dns records for my website www.mywebsite.com point to Amazon's IP address, instead of my home address, resulting in my websites ip address being that of Amazon's IP, as opposed to my Home IP.

**STEPS I HAVE TAKEN:**

1. So far, I have installed openvpn client on my webserver, and successfully tunneled it to amazons AWS EC2 OpenVPN Access Server, using openvpnclient on the webserver/email server. There are currently no DNS leaks on my webserver, and my public ip address and DNS address on the web server both point to Amazon's corresponding public IP and DNS.
2. I have changed my DNS records on domain registrars website, to point to the newly created AMAZON OpenvpnAS Public IP address.
3. I have opened all webserver (443,80) ports and email server (25,993,etc.) ports using UFW on the EC2 Instance.

**THE QUESTION:**

With all of the above settings set, My webserver/email server is still innaccessable when going to www.mywebsite.com. Now that my my server is being successfully tunneled as a client to the OpenVPNAS, and the DNS records are pointing to the EC2 IP address,

1. WHAT ELSE MUST I DO IN ORDER TO GET MY EMAIL SERVER/WEBSERVER TO PROPERLY RESOLVE THROUGH THE OPENVPN TUNNEL?

2. Should I be forwarding ports from the ec2 instance public IP to the vpn client, and if so, how exactly do I do such a thing on an openvpnas instance? The command line instructions to use DMZ forwarding errors out. So what how exactly can I forward the aws public ip and ports 443 and 80 to the client vpn webserver?

chilinux
OpenVPN Power User
Posts: 122
Joined: Thu Mar 28, 2013 8:31 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by chilinux » Fri Jul 09, 2021 6:13 am

It sounds like you have an EC2 instance that has only a single public IP that you expect to share back to the LEMP server.

If that is the case, then you will need to set up Destination Network Address Translation (DNAT) which would go in the PREROUTING/POSTROUTING chains of iptables.

If you have used UFW to create entries in the INPUT chain of iptables then that just allows the packets to services running on the immediate EC2 server rather than forwarded across the tunnel. It is DNAT instead that will perform the packet forwarding and IP translation you want.

There is some changes you should make in OpenVPN AS first.

Under: Configuration -> Network Settings -> Multi-Daemon Mode -> TCP port number
Change the port from 443 to a different port that you don't plan to switch to performing DNAT on.

Make sure whatever port you set is also allowed by UFW.

Next, Under: Configuration -> VPN Settings -> Routing -> Should client Internet traffic be routed through the VPN?
Make sure this is set to Yes

You may want to set the LEMP server's user account to have a consistent static intranet address.
This can be doing under User Management -> User Permissions -> [LEMP user name] -> More Settings -> IP Addressing -> Use Static
Then also supply the intranet IP under VPN Static IP Address

You should then be able to set up DNAT to/from the static VPN intranet address via UFW by following the examples given in this document:
https://bobcares.com/blog/ufw-port-forwarding/

Lastly, whenever you restart UFW, it may flush all the iptables rules including the ones OpenVPN AS expects to have in place to work. As such, every time you restart UFW, follow it by then restarting OpenVPN AS to allow it to re-establish it's own iptables entries.

User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Sat Jul 10, 2021 8:37 pm

This is extremely helpful, and thus far, I have done everything except for the two last steps (setting up DNAT and flushing the tables). I find the instructions at bobcares.com a bit vague. Maybe you can help me out here.

My opvnpnas public (amazon static) ip is

Code: Select all

n.nnn.nnn.nnn
Which is NOT LISTED in

Code: Select all

ifconfig
or

Code: Select all

ip addr show
On my openas server, the command

Code: Select all

ip addr show
returns:

Code: Select all

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000
    link/ether aa:aa:aa:aa:aa:aa brd ff:ff:ff:ff:ff:ff
    inet bbb.bb.b.bbb/20 brd ccc.cc.cc.ccc scope global dynamic eth0
       valid_lft 3367sec preferred_lft 3367sec
    inet6 fe90::d8:11ff:fb20:296c/64 scope link
       valid_lft forever preferred_lft forever
5: as0t0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 200
    link/none
    inet 173.21.223.1/20 scope global as0t0
       valid_lft forever preferred_lft forever
    inet6 fb70::2010:f7fc:849d:8380/64 scope link stable-privacy
       valid_lft forever preferred_lft forever
On my vpn client (the web server), I am successfully connected to my EC2 Openvnpas Server and the command

Code: Select all

ip addr show
returns:

Code: Select all

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether yy:yy:yy:yy:yy:yy brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.20/24 brd 192.168.30.255 scope global dynamic wlan0
       valid_lft 863798sec preferred_lft 863798sec
    inet6 fe70::bcd5:22ff:fbe5:32d2/64 scope link
       valid_lft forever preferred_lft forever
4: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    link/none
    inet 10.8.0.20/24 brd 10.8.0.255 scope global tun0
       valid_lft forever preferred_lft forever
On the tutorial that you instructed me to use, it instructs you to put the following rules in

Code: Select all

/etc/ufw/before.rules
.

Code: Select all

*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -i eth0 -d 2xx.2xx.1xx.0 -p tcp --dport 80 -j DNAT --to-destination 1xx.1xx.2xx.0:8080
# setup routing
-A POSTROUTING -s 1xx.1xx.2xx.0/24 ! -d 1xx.1xx.2xx.0/24 -j MASQUERADE
COMMIT
But it doesn't explain which ip addresses I am supposed to enter for the multiple (4 to be exact) ip address entries listed in the example. What should I be entering there as a replacement of the examples ip addresses?

Lastly, does the order matter where these rules are supposed to go in /etc/ufw/before.rules? Should it be at the top of the page, the bottom of the page, or in betweeen something?

chilinux
OpenVPN Power User
Posts: 122
Joined: Thu Mar 28, 2013 8:31 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by chilinux » Sun Jul 11, 2021 1:40 am

If I am reading your information correctly, you need to move packets arriving at bbb.bb.b.bbb to 10.8.0.20 and back again.

The example would become:

Code: Select all

*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -i eth0 -d bbb.bb.b.bbb -p tcp --dport 80 -j DNAT --to-destination 10.8.0.20:80
# setup routing
-A POSTROUTING -s 10.8.0.0/24 ! -d 10.8.0.0/24 -j MASQUERADE
COMMIT
You will need to repeat the PREROUTING rule for each port you want to forward (443, 25, 993, etc).

If you find iptables confusing/frustrating, you can accomplish the same goal by running NGINX on the EC2 server to act as the reverse proxy.

Since you are already learning NGINX on the LEMP server, you might find that easier to configure than iptables.

xinetd also provides reverse proxy forwarding.

User avatar
openvpn_inc
OpenVPN Inc.
Posts: 330
Joined: Tue Feb 16, 2021 10:41 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by openvpn_inc » Mon Jul 12, 2021 10:14 am

Hello Danran,

> 2. Should I be forwarding ports from the ec2 instance public IP to the vpn client, and if so, how exactly do I do such a thing on an openvpnas instance? The command line instructions to use DMZ forwarding errors out. So what how exactly can I forward the aws public ip and ports 443 and 80 to the client vpn webserver?

Yes that's what you should do. You can use the DMZ function to forward ports on the IP of your EC2 instance to your connected VPN client. Of course with AWS you have to keep in mind that its external public IP address is not actually on the EC2 instance. It's a 1:1 NAT to the private IP address of your EC2 instance, so in this case you need to specify just the local private IP of the EC2 instance that your Access Server is on - not the attached public IP.

See this document. This has been tested by me today to work with OpenVPN Access Server 2.9.2 on Amazon AWS:
https://openvpn.net/vpn-server-resource ... ss-server/

You have to only do these instructions to make this work. None of the other things mentioned in this topic are necessary to achieve this goal. The DMZ functionality is a standard function built into Access Server.

If you run into any problems with this please let us know the error message you got.

Kind regards,
Johan
Image OpenVPN Inc.
Answers provided by OpenVPN Inc. staff members here are provided on a voluntary best-effort basis, and no rights can be claimed on the basis of answers posted in this public forum. If you wish to get official support from OpenVPN Inc. please use the official support ticket system: https://openvpn.net/support

chilinux
OpenVPN Power User
Posts: 122
Joined: Thu Mar 28, 2013 8:31 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by chilinux » Tue Jul 13, 2021 7:20 pm

If the client DMZ shares the same IP address that OpenVPN AS is bound to, what happens with port 443 when the DMZ user is connected?

Does it continue to go to OpenVPN AS or is it forwarded on to the LEMP server?

If it is forwarded on to the LEMP server, shouldn't OpenVPN AS stop making references to TCP 443 in the client ovpn configuration files it generates?

User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Wed Jul 14, 2021 6:21 pm

openvpn_inc wrote:
Mon Jul 12, 2021 10:14 am
Hello Danran,

> 2. Should I be forwarding ports from the ec2 instance public IP to the vpn client, and if so, how exactly do I do such a thing on an openvpnas instance? The command line instructions to use DMZ forwarding errors out. So what how exactly can I forward the aws public ip and ports 443 and 80 to the client vpn webserver?

Yes that's what you should do. You can use the DMZ function to forward ports on the IP of your EC2 instance to your connected VPN client. Of course with AWS you have to keep in mind that its external public IP address is not actually on the EC2 instance. It's a 1:1 NAT to the private IP address of your EC2 instance, so in this case you need to specify just the local private IP of the EC2 instance that your Access Server is on - not the attached public IP.

See this document. This has been tested by me today to work with OpenVPN Access Server 2.9.2 on Amazon AWS:
https://openvpn.net/vpn-server-resource ... ss-server/

You have to only do these instructions to make this work. None of the other things mentioned in this topic are necessary to achieve this goal. The DMZ functionality is a standard function built into Access Server.

If you run into any problems with this please let us know the error message you got.

Kind regards,
Johan
So in Openvpnas web interface>user management>user permissions> myuser> more settings...
I put...
myamazonec2privateip:tcp/443
myamazonec2privateip:tcp/80

and my webserver is now properly working and serving my site on the EC2 instances vpn tunnel! THANK YOU BOTH FOR THE HELP!

Before I mark the thread as solve, I just want to verify a few more things. First, would be the question above from user "chilinux".

Then, I also noticed that my Amazon EC2 instance's private IP is the same ip as my EC2 instances eth0 interface. Does that sound right to you? Lastly, to double check, I do not need to set any firewall rules or do anything else after this, correct? (Maybe off topic) Is there anything in terms of security or functionality that you can think of that should be used on an ec2 openvpnas instance? I already installed fail2ban, but only have it set up for ssh. The only thing I noticed Is my web server is running relatively slow off of the vpn instance, but I'm assuming there is no way of speeding this up and this is due to the natural lag of the openvpn?

chilinux
OpenVPN Power User
Posts: 122
Joined: Thu Mar 28, 2013 8:31 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by chilinux » Thu Jul 15, 2021 8:01 pm

Your EC2 eth0 should be set to the instance private IP. That is normal.

Also, the DMZ feature should allow OpenVPN AS to take care of all the iptables rules for you. Changing UFW shouldn't be required.

Other security hardening I would recommend:

Move away from using passwords to authenticate SSH sessions. Instead use ssh-keygen and authorized_keys file to perform key based authentication. Once you confirmed you can authenticate via a ssh key, modify the sshd_config to set PasswordAuthentication to no. After making the modification, restart the ssh server. Once you get rid of password based ssh authentication, I personally see no reason to continue to use fail2ban.

Additional hardening beyond that would depend on if this is a production or a test/hobby instance of OpenVPN AS. If you are using this as a production server then do *NOT* proceed to the next suggestion. Otherwise, for a test instance, you can try playing with turning on SElinux. Doing this goes against the official documentation of OpenVPN AS. Depending on what features you use or what changes are made in future updates to OpenVPN AS, you need to be prepared to audit the events SElinux prohibits and update the SElinux policies accordingly.

I have been trying OpenVPN AS on CentOS 7 with SElinux enabled on a *test* system. So far I haven't run into any problems. But for a test system, I see problems with SElinux as a learning opportunity and consider SElinux something worth learning for anyone interested in server hardening.

To be clear (if I wasn't already), I am only recommending this for a *test* system with expectations that functionality may break and that SElinux is outside of the scope of OpenVPN AS support. For a production system, enabling SElinux is not recommended.

As far as performance, it is hard to tell without knowing the EC2 instance type you are using. For example, with a c4.large instance has AES-NI support built into the hardware and OpenVPN overhead should be minimal. Running "openssl speed" on different instance types should give you an idea of the level of overhead encrypting/decrypting traffic should have on that instance.

User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Thu Sep 16, 2021 8:56 am

So I think I need to re-open this thread, as I am still having issues regarding this post. I HAVE set up DMZ forwarding on the AWS Instance, and the WEB SERVER is serving properly. However, I seemed to overlook one big obstacle with all of this. I neglected to mention that the Web Server is also a postfix/dovecot email server as well. The reason for my neglect, is that I assumed that if my web server worked accross the VPN, that my email server would do just the same. However, I have hit a serious wall when it comes to the EMAIL/WEBSERVER receiving emails from an outside source such as gmail on a different computer.

To start/continue, it seems as though (according to answers from https://community.linuxbabe.com/t/email ... 4?u=danran and https://serverfault.com/questions/50313 ... hrough-vpn) my VPS (running openvpn-as) doesn't forward the public IP address of gmail to my mail/web server, and my mail server is only seeeing the IP address of the VPS (10.10.9.1).

I need to find a way for my VPS to forward the Public IP address of my gmail to my email/web server, and for my webserver to recognize that forwarded Public IP address. As of now, my email server is only seeing 10.10.9.1.

You can see how postfix is denying my emails due to the IP 10.10.9.1 not having a reverse hostname (which it shouldn't) in the postfix logs below:

Code: Select all

Sep 15 09:37:14 mail postfix/smtpd[4323]: NOQUEUE: reject: RCPT from unknown[10.10.9.1]: 450 4.7.1 Client host rejected: cannot find your reverse hostname, [10.10.9.1]; from=<danran@gmail.com> to=<admin@facl.xyz> proto=ESMTP helo=<mail-io1-f54.google.com>
Sep 15 09:37:14 mail postfix/smtpd[4323]: disconnect from unknown[10.10.9.1] ehlo=2 starttls=1 mail=1 rcpt=0/1 bdat=0/1 quit=1 commands=5/7
The author of linuxbabe.com has suggested that I set up an SMTP & IMAP Proxy with HAProxy, but I am almost certain their must be a cleaner solution that can be done strictly with openvpn on my VPS.

Could anyone provide some help in understanding what is going on here, and what I need to do exactly to fix this?

Thanks in advance for any help!

User avatar
openvpn_inc
OpenVPN Inc.
Posts: 330
Joined: Tue Feb 16, 2021 10:41 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by openvpn_inc » Thu Sep 16, 2021 2:39 pm

Hello Danran,

I tend to agree with the assessment that setting up an SMTP proxy on the Access Server itself to run mail operations is a reasonable solution. It would be actually using the real IP, and be a front to the outside world, while in the backend your mailserver connected via VPN is handling stuff. For IMAP I don't know if that's such a great solution but it could work.

The VPN client machine has a private IP, it does not have a public IP. A private IP cannot be routed on the Internet. So therefore NAT must be applied. The way Access Server does it is by replacing the source and destination IP addresses, so that the traffic can be handled within the VPN subnet, and replies in and out go through the Access Server's public IP. This is done because we cannot assume that the VPN tunnel is doing full redirection of traffic, it could be just split-tunnel. And then you'd get asymmetric routing which would break things if request from outside came in and your VPN client tried to respond to it and it's not forwarded back through the VPN tunnel.

What you could try is to turn off the functionality that Access Server implements for DMZ, the ability to forward traffic coming on a public IP on the Access Server and NATting it to a VPN client. And then you ensure this VPN client has a static IP in the VPN network, and then you create your own iptables NAT rule. One that does NAT outgoing but does not NAT incoming. Basically the source address would remain intact and the VPN client can see the original IP. However this does require that the VPN client is configured in such a way that it sends all its Internet traffic through the VPN tunnel, or else it will fail.

Access server already does the outgoing NAT. You just need to forward traffic coming in on certain ports on the public IP of the Access Server to a specific VPN internal IP of this VPN client, without doing it with NAT. So just a port forward. The Access Server has no configuration option for this, you'd have to do it yourself in iptables.

Kind regards,
Johan
Image OpenVPN Inc.
Answers provided by OpenVPN Inc. staff members here are provided on a voluntary best-effort basis, and no rights can be claimed on the basis of answers posted in this public forum. If you wish to get official support from OpenVPN Inc. please use the official support ticket system: https://openvpn.net/support

User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Thu Sep 16, 2021 4:38 pm

Thank you for such a prompt reply! However, I am having a bit of difficulty grasping this. Let me try to summarize here....
wrote: What you could try is to turn off the functionality that Access Server implements for DMZ, the ability to forward traffic coming on a public IP on the Access Server and NATting it to a VPN client.
The sentance structure is a bit confusing to me here. What exactly do you mean? That I need to somehow turn off what access server does with default dmz forwarding rules, by using iptables? In other words, I need to stop access server from forwarding the incoming traffic to the vpn, and instead NAT it to the VPN Cleint? I dunno. Would you remind revising that sentance? I'm not grasping it.
And then you ensure this VPN client has a static IP in the VPN network,
My VPN client does have a static IP assigned to it by the access server, if that is what you mean.
and then you create your own iptables NAT rule.
This makes me ask, where does Openvpn-AS Store its default iptables NAT rules? How can I ovveride those rules to create my own NAT rules?
One that does NAT outgoing but does not NAT incoming. Basically the source address would remain intact and the VPN client can see the original IP. However this does require that the VPN client is configured in such a way that it sends all its Internet traffic through the VPN tunnel, or else it will fail.
This is not a problem for me, I absolutely want all of my traffic being tunneled through the vpn. No split tunnels for me.
Access server already does the outgoing NAT. You just need to forward traffic coming in on certain ports on the public IP of the Access Server to a specific VPN internal IP of this VPN client, without doing it with NAT.
What do you mean by a "specific VPN internal IP of this VPN client"? and "without doing it with NAT"? I'm totally confused here. Do you have some examples, or could you possibly revise your sentance structure?
So just a port forward. The Access Server has no configuration option for this, you'd have to do it yourself in iptables.
To me, it sounds like you are saying that basically I am already set up and ready to go, except for port forwarding. So all I have to do is create a custom IPTABLES rule that forwards ports from my AS Public IP address, to my VPN Static Client? If so, can you give me a good example of a rule? I am very new to IPTABLES and generally stick to using UFW as my firewall.

It seems like I only have a small grasp (if any) on what you are sayin here in full. Could you please correct me where my assumptions have gone wrong here? Any further help getting me to obtain a grasp on this is certainly much appreciated.

Thank you Johan!

User avatar
openvpn_inc
OpenVPN Inc.
Posts: 330
Joined: Tue Feb 16, 2021 10:41 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by openvpn_inc » Fri Sep 17, 2021 2:29 pm

Hello,

When you use the DMZ function in Access Server, it forwards traffic and it uses NAT. That means the source IP of requests coming in changes. And that means you don't see the original IP of the requests coming from outside. And that's your problem. So the idea is to turn off DMZ function in Access Server. And then implement your own solution in iptables. But do so without NAT. So that the original source IP remains intact.

You can find more about iptables related settings in Access Server here:
https://openvpn.net/faq/how-do-i-preven ... inux-host/

Please note that we do not extend our support to custom iptables.

Kind regards,
Johan
Image OpenVPN Inc.
Answers provided by OpenVPN Inc. staff members here are provided on a voluntary best-effort basis, and no rights can be claimed on the basis of answers posted in this public forum. If you wish to get official support from OpenVPN Inc. please use the official support ticket system: https://openvpn.net/support

User avatar
Danran
OpenVPN User
Posts: 25
Joined: Tue Jun 29, 2021 9:21 am

Re: LEMP Webserver as OpenVPN client to aws instance - Proper port forwarding?

Post by Danran » Fri Sep 17, 2021 4:07 pm

This is much more clear! Thank you for the answer. I understand now!

Post Reply