TLS Error / NAT Two Remote statements

reberhar

Hi Guys,

Every time I start a new forum I have to learn how to start a new topic, find the right forum and the right way to do things etc. This is my first try on OpenVPN so I am expecting some tutoring from you all.

First I am running pfsense in a CARP / High Availability arrangement which works very well. I am using the pfsense utilities for my OpenVPN setup. We have OpenVPN tunnels running between our servers which also work very well. My pfsenses are clients in these. I also have my own OpenVPN server installed which was working very well until the cable provider decided to change the modem. So now we are behind their nat. I have managed to get it to work anyway, I just need some help understanding how to clean it up a bit. I am using dyndns in two ways. I am resolving the visible external Internet address that dyndns says is 189.215.XXX.XXX. Of course that doesn't get me far enough. I am also resolving the internal address that I find in the modem 10.XXX.XXX.XXX. I am using two remote statements.

remote beforethenat.dyndns.org 1194 udp
remote behindthenat.dyndns.org 1194 udp

the modem has a virtual server for coming in and port triggering for going out. I probably don't need both but I don't think it hurts.

I am also using --float because this messes with the port number

the first remote statement fails the TLS handshaking.
the second remote succeeds but seems to use routing discovered by the first remote statement.

Here is the config.

dev tun
persist-tun
persist-key
cipher AES-256-CBC
ncp-ciphers AES-256-GCM:AES-128-GCM:AES-256-CBC
auth SHA1
tls-client
client
resolv-retry infinite
remote beforethenat.dyndns.org 1194 udp
remote behindthenat.dyndns.org 1194 udp
verify-x509-name "MY Office Certificate" name
auth-user-pass189.215.X
pkcs12 pfsensefirewallprimary-UDP4-1194-XXX.p12
tls-auth pfsensefirewallprimary-UDP4-1194-XXX-tls.key 1
remote-cert-tls server
passtos

# script-security 2 $
# up /etc/openvpn/update-resolv-conf
# down /etc/openvpn/update-resolv-conf

Here is the client log file.

XXX@XXX-CrucialHD:/etc/openvpn$ sudo openvpn --float --config pfsensefirewallprimary-UDP4-1194-XXX-config.ovpn
Fri Jul 27 10:34:09 2018 WARNING: file 'pfsensefirewallprimary-UDP4-1194-XXX.p12' is group or others accessible
Fri Jul 27 10:34:09 2018 WARNING: file 'pfsensefirewallprimary-UDP4-1194-XXX-tls.key' is group or others accessible
Fri Jul 27 10:34:09 2018 OpenVPN 2.4.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Feb 10 2018
Fri Jul 27 10:34:09 2018 library versions: OpenSSL 1.1.0g 2 Nov 2017, LZO 2.08
Enter Auth Username: XXX
Enter Auth Password: *****************
Enter Private Key Password: *********
Fri Jul 27 10:34:27 2018 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Fri Jul 27 10:34:37 2018 TCP/UDP: Preserving recently used remote address: [AF_INET]189.XXX.XXX.XXX:1194
Fri Jul 27 10:34:37 2018 UDP link local (bound): [AF_INET][undef]:1194
Fri Jul 27 10:34:37 2018 UDP link remote: [AF_INET]189.XXX.XXX.XXX:1194
Fri Jul 27 10:35:38 2018 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Fri Jul 27 10:35:38 2018 TLS Error: TLS handshake failed
Fri Jul 27 10:35:38 2018 SIGUSR1[soft,tls-error] received, process restarting
Fri Jul 27 10:35:52 2018 TCP/UDP: Preserving recently used remote address: [AF_INET]10.XXX.XXX.XXX:1194
Fri Jul 27 10:35:52 2018 UDP link local (bound): [AF_INET][undef]:1194
Fri Jul 27 10:35:52 2018 UDP link remote: [AF_INET]10.XXX.XXX.XXX:1194
Fri Jul 27 10:35:58 2018 [MY Office Certificate] Peer Connection Initiated with [AF_INET]189.XXX.XXX.XXX:36620
Fri Jul 27 10:36:00 2018 Options error: Unrecognized option or missing or extraI will include the parameter(s) in [PUSH-OPTIONS]:6: register-dns (2.4.4)
Fri Jul 27 10:36:00 2018 TUN/TAP device tun0 opened
Fri Jul 27 10:36:00 2018 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Fri Jul 27 10:36:00 2018 /sbin/ip link set dev tun0 up mtu 1500
Fri Jul 27 10:36:00 2018 /sbin/ip addr add dev tun0 192.168.160.2/24 broadcast 192.168.160.255
Fri Jul 27 10:36:00 2018 Initialization Sequence Completed

Getting the log file from the server is problematic because I am connected through the OpenVPN tunnel. The other tunnel wipes out the log on sequence logs before I can get to them. I must log out to create them. By the time I am reconnected they are gone.

e.g.
Jul 27 12:18:39 openvpn 44198 MANAGEMENT: Client disconnected
Jul 27 12:18:39 openvpn 44198 MANAGEMENT: CMD 'status 2'
Jul 27 12:18:39 openvpn 44198 MANAGEMENT: CMD 'state 1'
Jul 27 12:18:39 openvpn 44198 MANAGEMENT: Client connected from /var/etc/openvpn/client2.sock
Jul 27 12:17:52 openvpn 44198 MANAGEMENT: Client disconnected
Jul 27 12:17:52 openvpn 44198 MANAGEMENT: CMD 'status 2'
Jul 27 12:17:52 openvpn 44198 MANAGEMENT: CMD 'state 1'
Jul 27 12:17:52 openvpn 44198 MANAGEMENT: Client connected from /var/etc/openvpn/client2.sock
Jul 27 12:17:50 openvpn 44198 MANAGEMENT: Client disconnected
Jul 27 12:17:50 openvpn 44198 MANAGEMENT: CMD 'status 2'
Jul 27 12:17:50 openvpn 44198 MANAGEMENT: CMD 'state 1'
Jul 27 12:17:50 openvpn 44198 MANAGEMENT: Client connected from /var/etc/openvpn/client2.sock
Jul 27 12:17:06 openvpn 44198 MANAGEMENT: Client disconnected
Jul 27 12:17:06 openvpn 44198 MANAGEMENT: CMD 'status 2'
Jul 27 12:17:06 openvpn 44198 MANAGEMENT: CMD 'state 1'
Jul 27 12:17:06 openvpn 44198 MANAGEMENT: Client connected from /var/etc/openvpn/client2.sock
Jul 27 12:16:17 openvpn 44198 MANAGEMENT: Client disconnected

The routing table only includes the external address and the 192.168.160.X addresses.

As always this seems overwhelming until I have dominated it and sorted out the details.

Any help you can provide is appreciated, both in using the forum and in cleaning up my configuration.

Thanks,

reberhar

TinCanTech

We really do need to see the server log.

However,

reberhar wrote Fri Jul 27 10:34:37 2018 TCP/UDP: Preserving recently used remote address: [AF_INET]189.XXX.XXX.XXX:1194
Fri Jul 27 10:34:37 2018 UDP link local (bound): [AF_INET][undef]:1194
Fri Jul 27 10:34:37 2018 UDP link remote: [AF_INET]189.XXX.XXX.XXX:1194
Fri Jul 27 10:35:38 2018 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Fri Jul 27 10:35:38 2018 TLS Error: TLS handshake failed
Fri Jul 27 10:35:38 2018 SIGUSR1[soft,tls-error] received, process restarting
Fri Jul 27 10:35:52 2018 TCP/UDP: Preserving recently used remote address: [AF_INET]10.XXX.XXX.XXX:1194
Fri Jul 27 10:35:52 2018 UDP link local (bound): [AF_INET][undef]:1194
Fri Jul 27 10:35:52 2018 UDP link remote: [AF_INET]10.XXX.XXX.XXX:1194
Fri Jul 27 10:35:58 2018 [MY Office Certificate] Peer Connection Initiated with [AF_INET]189.XXX.XXX.XXX:36620

this does not make any sense to me.

You cannot route a 10/8 IP address over the Internet

reberhar

Yes exactly. 10.x.x.x is an internal, never mind the /8. I put the xxxs in for privacy. Yet as you can see by the log, it connects. The 10.x.x.x is the NATed address from the service provider in the modem on my side.

Using two remotes, one with the external and one with the internal and --float gets me to the server. I really don't want to bridge if I don't have to. I have two servers in a high definition CARP arrangement. I need two IPs, but in bridge mode I only get one. I can use a router but I don't really want to add another piece of hardware.

I did find the "How to" on submitting a new thread. I have all the documents you ask for. I just have to edit out the personal stuff. At verb 4 that's a lot. The server log only mentions the external IP. Everything seems to be happening at the client level. I am not an OpenVPN expert so that is only a guess.

I hope to submit a complete thread Tuesday.

Thanks

TinCanTech

reberhar wrote I hope to submit a complete thread Tuesday.

No need .. this is not an openvpn problem.

This is the problem:

reberhar wrote So now we are behind their nat. I have managed to get it to work anyway,

I do not belive you have because you cannot connect from here:

Fri Jul 27 10:34:37 2018 UDP link remote: [AF_INET]189.XXX.XXX.XXX:1194
Fri Jul 27 10:35:38 2018 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Fri Jul 27 10:35:38 2018 TLS Error: TLS handshake failed
Fri

reberhar

I guess you can believe what you want. I am connected this minute.
The first remote fails.
The second remote succeeds.

Sun Jul 29 17:44:33 2018 [ILV Santarosa Office Certificate] Peer Connection Initiated with [AF_INET]189

Without both of these and float there is no success.

TinCanTech

Try connecting from a physically remote location ..

reberhar

I am at a physically remote location right now.

I am connecting over the Internet.

🙂

reberhar

The OpenVPN client software seems to be quite sophisticated. If I controlC out as I have to in Linux, on reconnect it remembers the previous certificate, IP and synchronization. It seems to know and use both the external Internet address that I give it and the internal natted address as well, allowing it to connect. I first discovered this by running two different config files. I can run one config with the external and a second with the internal and the second connects. The idea of multiple remotes occurred to me later. I have been trying to read the documentation on this but it is of course typically cryptic until you get a handle on it. The only thing I find on natting is the --float command. That does not mean that the information I see is not there. It just means I haven't recognized it. I looked at snat and dnat, but that is hard to distribute and it doesn't seem to be what I want.

TinCanTech

Please post your client log showing the ip of the server (at least the first octet)

reberhar

Tue Jul 31 10:45:45 2018 [Ny Office Certificate] Peer Connection Initiated with [AF_INET]189.x.x.x:29996

The OpenVPN server is over 4 miles away.

TinCanTech

Indeed .. not the 10.x.x.x address

reberhar

I did an nmap from the pfsense box. The service provider has lots of folks out there on that internal. I think I am safe though. The firewall blocks access from such addresses on the WAN side.

reberhar

Look, if this is a pain in the neck for you I will just tweak my setup to my liking, lowering the error timeout from one minute to 5 seconds or so. I realize this should not work. Perhaps this is just an unusual set of circumstances that allows this to happen. You don't have to dink with this. I just thought there might be some way to set it up to avoid the failed first connection.

TinCanTech

reberhar wrote I just thought there might be some way to set it up to avoid the failed first connection.

The only connection that should work is this one:

reberhar wrote Tue Jul 31 10:45:45 2018 [Ny Office Certificate] Peer Connection Initiated with [AF_INET]189.x.x.x:29996

Other than that, I have no idea how your network or your ISP network is set up. Maybe you can ask them what you should expect ..

reberhar

Thanks for your help on this. I note that I have the same service provider on both ends of this tunnel. I am bridged on the remote end. I will take it to a place where I am entering through a different service provider. If it works then, it is a true puzzle.

I am going to look at the IPs and the masks and see how this is sneaking through.

The ISP technical support is a difficult online Dr Fixit and not real people. Your support has been ever so much better.
In the end, I will probably have to byte the bullet and do a bridge. Nuts.

You have been great. Thanks so much.

Pippin

So you`re in a CGN?
I wonder, pfSense`s state table?