Site to Site communication issues...
Posted: Mon Sep 23, 2019 4:53 pm
I'm trying to setup a site to site vpn with openvpn at both ends. This has been done before and on the surface it seems straight forward, but I guess I'm trying to do things a little weird. My company has a cluster of disparate sites with a range of network products making a uniform vpn solution rather difficult and we struggle with outages ALL the dang time. the biggest problem I'm facing is the routers. some are sonicwall, some ubiquiti, and some are cisco. None of them can have a stable s2s vpn with all the others simultaneously. the plan is to install openvpn servers at each site, BEHIND the routers and just link the sites via tunnels inside the local LANs with a set of static routes pointing back to the VPN servers for LAN2LAN communication.
right now, I'm working on one link as a proof of concept.
the layout is thus (fake IPs are fake):
Site 1
Site 2
So right off, you'll notice site 1 has vlans that will obfuscate things a bit. the VPN server and router are in vlan 21 while the clients are in vlan 25. not sure if this is a problem, but it's something to make note of.
Next off, the vpn server at site 1 is running two instances of openvpn. One for remote connections from remote users (it's a generic setup) and a second running for site 2 site.
Now for the setup, I've used a script from this site to both install and configure openvpn on server1. it's a good script and I've used it at numerous sites to get things up and running quickly. plus it lets me setup new users quick, fast, and in a hurry.
I used said script to generate a new client config for a client called "s2sclient". Then I copied the original server.conf file to a new config file called s2sserver.conf and changed it's udp port and IP config (using 1195 and 10.1.0.0 instead of 10.8.0.0) so it wouldn't conflict with openvpn@server so now I have openvpn@server and openvpn@s2sserver.
Next I updated /etc/iptables/add-openvpn-rules.sh thusly:
Next I rebooted the server and checked that my normal remote clients can still connect... Good so far!
Now, the vpn server at site 2 is running windows 2k12 so I just installed the openvpn community client for win7 (as suggested) and imported the s2sclient.ovpn. Connect... and the connection is successful, I have a basic vpn from the site 2 vpn server to site 1 and I have a stable connection... sort of... I am also having the issue described here and here and here... yeah... it's a pretty common problem. but it seems to be a problem with the OS and I don't care to address it here. Running ipconfig /renew "tun1" gets me around it.
At this point I can ping and connect to most network devices on the site1 network from the site2 vpn server and vice versa. so now I need to get routing up and running. For this I adapted the guide here. Basically all I did was run the following commands:
...add this to the s2sserver.conf:
then restart the s2sserver and client...
NOTE: I have 4 of the 6 routes commented out because the tunnels for vlans 21 and 22 don't work on the existing VPN anyway and therefore won't cause routing conflicts, while vlans 23-26 DO work and might gum things up if suddenly a second route for each appears. Once I have 21 and 22 working, I'll enable the other 4 and bring down the existing crap-tastic vpn.
From here, things appear to be setup correctly. from both the client and server, I am still able to ping and connect to network resources on the other side of the vpn without incident. however, this is where things stop making progress...
on both ends I have a test client. I assume that if I change my test client's default gateway to point to the VPN server at either location, I should be able to see remote resources... right? ...because I can't. while each vpn server is able to connect to the other and browse network resources, none of the clients behind them can. I even went so far as to disable the firewall on the win2k12 server to make sure it wasn't a firewall issue, and as you can see the firewall (iptables) on the linux server is set to allow all traffic through those tunnels... so what am I missing?
on the win2k12 side I checked the logs to be sure the routes are pushing and I see the following:
As noted above, I can see the two desired routes for 192.168.21.0 and 192.168.22.0... so what gives?
I thought it might be the ccd, but if that were the case then I'd expect the clients at site1 to not see site2 resources, but then site2 clients should still be able to see site1 resources... right?
right now, I'm working on one link as a proof of concept.
the layout is thus (fake IPs are fake):
Site 1
Code: Select all
clients
- LAN: 192.168.25.255/24
vpnserver1
- LAN: 192.168.21.10/24
EdgeRouter1
- LAN: 192.168.21.1/24
-WAN: 11.0.0.1/32
Code: Select all
clients
-LAN: 192.168.0.255/24
vpnserver2
-LAN: 192.168.0.12/24
SonicWall2
-LAN: 192.168.0.1/24
-WAN: 12.0.0.1/32
Next off, the vpn server at site 1 is running two instances of openvpn. One for remote connections from remote users (it's a generic setup) and a second running for site 2 site.
Now for the setup, I've used a script from this site to both install and configure openvpn on server1. it's a good script and I've used it at numerous sites to get things up and running quickly. plus it lets me setup new users quick, fast, and in a hurry.
I used said script to generate a new client config for a client called "s2sclient". Then I copied the original server.conf file to a new config file called s2sserver.conf and changed it's udp port and IP config (using 1195 and 10.1.0.0 instead of 10.8.0.0) so it wouldn't conflict with openvpn@server so now I have openvpn@server and openvpn@s2sserver.
Next I updated /etc/iptables/add-openvpn-rules.sh thusly:
Code: Select all
#!/bin/sh
#tun0 rules
iptables -t nat -I POSTROUTING 1 -s 10.8.0.0/24 -o ens192 -j MASQUERADE
iptables -I INPUT 1 -i tun0 -j ACCEPT
iptables -I FORWARD 1 -i ens192 -o tun0 -j ACCEPT
iptables -I FORWARD 1 -i tun0 -o ens192 -j ACCEPT
iptables -I INPUT 1 -i ens192 -p udp --dport 1194 -j ACCEPT
#s2s tun1 rules
iptables -t nat -I POSTROUTING 1 -s 10.1.0.0/24 -o ens192 -j MASQUERADE
iptables -I INPUT 1 -i tun1 -j ACCEPT
iptables -I FORWARD 1 -i ens192 -o tun1 -j ACCEPT
iptables -I FORWARD 1 -i tun1 -o ens192 -j ACCEPT
iptables -I INPUT 1 -i ens192 -p udp --dport 1195 -j ACCEPT
Now, the vpn server at site 2 is running windows 2k12 so I just installed the openvpn community client for win7 (as suggested) and imported the s2sclient.ovpn. Connect... and the connection is successful, I have a basic vpn from the site 2 vpn server to site 1 and I have a stable connection... sort of... I am also having the issue described here and here and here... yeah... it's a pretty common problem. but it seems to be a problem with the OS and I don't care to address it here. Running ipconfig /renew "tun1" gets me around it.
At this point I can ping and connect to most network devices on the site1 network from the site2 vpn server and vice versa. so now I need to get routing up and running. For this I adapted the guide here. Basically all I did was run the following commands:
Code: Select all
# mkdir /etc/openvpn/s2s1ccd
# touch /etc/openvpn/s2s1ccd/client
# echo “iroute 192.168.0.12 255.255.255.0” > /etc/openvpn/s2s1ccd/client
Server Config
#S2S routing
push "route 192.168.21.0 255.255.255.0"
push "route 192.168.22.0 255.255.255.0"
#push "route 192.168.23.0 255.255.255.0"
#push "route 192.168.24.0 255.255.255.0"
#push "route 192.168.25.0 255.255.255.0"
#push "route 192.168.26.0 255.255.255.0"
client-config-dir s2s1ccd
route 192.168.0.12 255.255.255.0
#/S2S routing
push "route 192.168.21.0 255.255.255.0"
push "route 192.168.22.0 255.255.255.0"
#push "route 192.168.23.0 255.255.255.0"
#push "route 192.168.24.0 255.255.255.0"
#push "route 192.168.25.0 255.255.255.0"
#push "route 192.168.26.0 255.255.255.0"
client-config-dir s2s1ccd
route 192.168.0.12 255.255.255.0
#/S2S routing
then restart the s2sserver and client...
NOTE: I have 4 of the 6 routes commented out because the tunnels for vlans 21 and 22 don't work on the existing VPN anyway and therefore won't cause routing conflicts, while vlans 23-26 DO work and might gum things up if suddenly a second route for each appears. Once I have 21 and 22 working, I'll enable the other 4 and bring down the existing crap-tastic vpn.
From here, things appear to be setup correctly. from both the client and server, I am still able to ping and connect to network resources on the other side of the vpn without incident. however, this is where things stop making progress...
on both ends I have a test client. I assume that if I change my test client's default gateway to point to the VPN server at either location, I should be able to see remote resources... right? ...because I can't. while each vpn server is able to connect to the other and browse network resources, none of the clients behind them can. I even went so far as to disable the firewall on the win2k12 server to make sure it wasn't a firewall issue, and as you can see the firewall (iptables) on the linux server is set to allow all traffic through those tunnels... so what am I missing?
on the win2k12 side I checked the logs to be sure the routes are pushing and I see the following:
Code: Select all
Mon Sep 23 09:59:59 2019 us=493445 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 192.168.24.18,dhcp-option DNS 192.168.0.12,redirect-gateway def1 bypass-dhcp,route 192.168.21.0 255.255.255.0,route 192.168.22.0 255.255.255.0,route-gateway 10.1.0.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.1.0.2 255.255.255.0,peer-id 1,cipher AES-128-GCM'
I thought it might be the ccd, but if that were the case then I'd expect the clients at site1 to not see site2 resources, but then site2 clients should still be able to see site1 resources... right?