Unfortunately, I don't know enough about the networking side to ask good questions, so for now the purpose of this post is just to find good docs. I have a GCE node on the [b]left [/b]side: External IP: 12.34.56.78 Internal IP: 10.254.0.2 Netmask: 10.255.255.255 (10.254.0.1 is used by the gateway in a GCE internal network, so I cannot use that address) It runs Centos 7, but I'm not married to that particular distro, so I'm happy to scrap it. My goal is to set up a site-to-site VPN to a BSD router (pfSense) on the [b]right [/b]side with: External IP: 87.65.43.21 Internal IP: 10.0.0.1 Netmask: 10.0.0.255 The ultimate end goal is to have multiple site-to-site VPNs up, with subnets like 10.30.0.0/16, 10.200.0.0/16, 10.80.0.0/16, etc, each obviously pointing to a unique LAN, with per subnet routing rules. I'd like to stay as close to the distro template's default settings, as they tend to come geared toward security out-of-the-box from Google. E.g. I'd rather not be disabling firewalld if I could avoid it, and I'd rather leave SElinux enabled unless it really complicates matters. Those things are secondary for now, however. (Note that I'm not entirely sure this is possible behind GCE. I've just been assuming. I'm aware that there is a site-to-site VPN solution baked into GCS, but it costs a few cents per hour, and multiplied by multiple client domains, many with multiple locations per domain, and it starts adding up to real money very quickly.) As I mentioned at the beginning of the post, I'm currently looking for docs. I working on a series of dependent assumptions in the search for docs. Currently I'm stuck at one of the early steps: setting up the bridged interface on the left side. I assume I want to set up a tap0 and br0 in order to accomplish that, but in the process of teaching myself how to do this, I've already managed to break the VM's networking twice, which is always a pain to recover from. [b]Any help with that step would be much appreciated[/b], as the sample configs and most of the documentation I have come across assumes this part is done, it seems.

[quote="FuriousGeorge"]for now the purpose of this post is just to find good docs[/quote]Did you read this: [list][*][url=http://openvpn.net/index.php/open-source/documentation.html]OpenVPN Community Edition Documentation[/url] [*][url=http://openvpn.net/index.php/open-source/documentation/howto.html]HOWTO: For OpenVPN Community Edition[/url][/list] [quote="FuriousGeorge"] setting up the bridged interface[/quote]And this: [list][*][url=http://openvpn.net/index.php/open-source/documentation/howto.html#vpntype]Determining whether to use a routed or bridged VPN[/url] [*][url=http://openvpn.net/index.php/open-source/documentation/miscellaneous/ethernet-bridging.html]Ethernet Bridging[/url][/list]

[quote="Traffic"][quote="FuriousGeorge"]for now the purpose of this post is just to find good docs[/quote]Did you read this: [list][*][url=http://openvpn.net/index.php/open-source/documentation.html]OpenVPN Community Edition Documentation[/url] [*][url=http://openvpn.net/index.php/open-source/documentation/howto.html]HOWTO: For OpenVPN Community Edition[/url][/list] [quote="FuriousGeorge"] setting up the bridged interface[/quote]And this: [list][*][url=http://openvpn.net/index.php/open-source/documentation/howto.html#vpntype]Determining whether to use a routed or bridged VPN[/url] [*][url=http://openvpn.net/index.php/open-source/documentation/miscellaneous/ethernet-bridging.html]Ethernet Bridging[/url][/list][/quote] Thanks. That was under my nose the whole time. Unfortunately, running bridge start will lock me out of my instance, and I'm not sure why. [code] #!/bin/sh ################################# # Set up Ethernet bridge on Linux # Requires: bridge-utils ################################# # Define Bridge Interface br="br0" # Define list of TAP interfaces to be bridged, # for example tap="tap0 tap1 tap2". tap="tap0" # Define physical ethernet interface to be bridged # with TAP interface(s) above. eth="eth0" eth_ip="10.254.0.2" eth_netmask="255.255.255.0" eth_broadcast="10.254.0.255" for t in $tap; do openvpn --mktun --dev $t done brctl addbr $br brctl addif $br $eth for t in $tap; do brctl addif $br $t done echo "got here 3" for t in $tap; do ifconfig $t 0.0.0.0 promisc up done ### This is where it fails ifconfig $eth 0.0.0.0 promisc up ifconfig $br $eth_ip netmask $eth_netmask broadcast $eth_broadcast [/code] I'm guessing it has something to do with the peculiarities of the GCE platform, and seeking support accordingly.

I've given up on the tap interface for the time being and I'm working on a tun based setup. Host to client I got it working after manually adding a route to the windows laptop. I was able to ping the server across the VPN. Then I tried to set up site-to-site. I can only ping my own remote IP on the OpenVPN server LAN from the client side. I can't ping any devices on the client side from the server side. I've tried adding the whole 10.0.0.0/8 range to the route on the pfsense (BSD) based router on the client side. Here is my server config: [code] port 1194 proto udp dev tun ca /etc/openvpn/keys/ca.crt cert /etc/openvpn/keys/server.crt key /etc/openvpn/keys/server.key # This file should be kept secret dh /etc/openvpn/keys/dh2048.pem server 10.254.254.0 255.255.255.0 ifconfig-pool-persist ipp.txt push "route 10.0.0.0 255.0.0.0" keepalive 10 120 comp-lzo persist-key persist-tun status openvpn-status.log ;log openvpn.log ;log-append openvpn.log verb 3 [/code] The Client Side is the pfsense GUI: [img] http://snag.gy/ums2C.jpg [/img] The server output looks good to me: [code] Wed Mar 16 07:32:46 2016 69.115.109.62:46450 TLS: Initial packet from [AF_INET]69.115.109.62:46450, sid=71f1b1b7aa100a87 Wed Mar 16 07:32:48 2016 69.115.109.62:46450 VERIFY OK: depth=1, C=US, ST=New Jersey, L=Newark, O=LALALALA, OU=MyOrganizationalUnit, CN=ads-vpn-server-1,name=EasyRSA,emailAddress=me@myemail.copm Wed Mar 16 07:32:48 2016 69.115.109.62:46450 VERIFY OK: depth=0, C=US, ST=New Jersey, L=Newark, O=LALALALA, OU=MyOrganizationalUnit, CN=ads-120elm-pfsense,name=EasyRSA,emailAddress=me@myemail.com Wed Mar 16 07:32:48 2016 69.115.109.62:46450 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 16 07:32:48 2016 69.115.109.62:46450 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 16 07:32:48 2016 69.115.109.62:46450 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key Wed Mar 16 07:32:48 2016 69.115.109.62:46450 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Wed Mar 16 07:32:48 2016 69.115.109.62:46450 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA Wed Mar 16 07:32:48 2016 69.115.109.62:46450 [ads-pfsense] Peer Connection Initiated with [AF_INET]69.115.109.62:46450 Wed Mar 16 07:32:48 2016 MULTI: new connection by client 'ads-pfsense' will cause previous active sessions by this client to be dropped. Remember to use the --duplicate-cn option if you want multiple clients using the same certificate or username to concurrently connect. Wed Mar 16 07:32:48 2016 MULTI_sva: pool returned IPv4=10.254.254.6, IPv6=(Not enabled) Wed Mar 16 07:32:48 2016 MULTI: Learn: 10.254.254.6 -> ads-pfsense/69.115.109.62:46450 Wed Mar 16 07:32:48 2016 MULTI: primary virtual IP for ads-pfsense/69.115.109.62:46450: 10.254.254.6 Wed Mar 16 07:32:51 2016 ads-pfsense/69.115.109.62:46450 PUSH: Received control message: 'PUSH_REQUEST' Wed Mar 16 07:32:51 2016 ads-pfsense/69.115.109.62:46450 send_push_reply(): safe_cap=940 Wed Mar 16 07:32:51 2016 ads-pfsense/69.115.109.62:46450 SENT CONTROL [ads-120elm-pfsense]: 'PUSH_REPLY,r oute 10.254.254.1,topology net30,ping 10,ping-restart 120,ifconfig 10.254.254.6 10.254.254.5' (status=1) [/code] Any help is much appreciated. I'll work on getting the br0 iface working for posterity.

[quote="FuriousGeorge"] I can't ping any devices on the client side from the server side[/quote]See this HOWTO: [url=http://openvpn.net/index.php/open-source/documentation/howto.html#scope]HOWTO: Expanding the scope of the VPN to include additional machines[/url] I strongly advise you read the [b]entire[/b] HOWTO .. [quote="FuriousGeorge"]Unfortunately, running bridge start will lock me out of my instance, and I'm not sure why.[/quote]Ensure your bridge has a gateway ..

I think I can help you with this. I'm also in similar setup, using a pfSense router as the VPN server. pfSense is well integrated with OpenVPN, so the options can be manipulated via GUI and it will be much easier. I also successfully made the site-to-site VPN link and really enjoying it. But first, two site's subnet address is colliding. GCD Node's 10.0.0.0/8 and pfSense's 10.0.0.0/24. GCD Node's subnet is including pfSense's subnet, as first 8 bits of address are same. GCD Node will think pfSense's subnet IP belong to its own subnet and try to send packets into its own subnet, not to pfSense. Unless you change one subnet into other address(For example, you may change pfSense's subnet to 192.168.xxx.0/24), you will not be able to achieve your goal.

I started with a fresh Centos 7.2 instance and disabled the firewall and selinux. Server IP: 10.250.0.2 Server Subnet: 10.250.0.0/16 Tun0 Interface IP: 10.254.254.1 Tun0 Interface Subnet: 10.254.254.0/24 Client IP: 10.0.0.1 Client Subset: 10.0.0.0/24 Here is server_tun.conf: [code] ;local a.b.c.d ;proto tcp proto udp ;dev tap0 dev tun ;dev-node MyTap ca /etc/openvpn/keys/ca.crt cert /etc/openvpn/keys/server.crt key /etc/openvpn/keys/server.key # This file should be kept secret dh /etc/openvpn/keys/dh2048.pem ;topology subnet server 10.254.254.0 255.255.255.0 ifconfig-pool-persist ipp.txt ;server-bridge 10.250.0.2 255.255.255.0 10.250.0.3 10.254.0.254 push "route 10.250.0.0 255.255.0.0" client-config-dir ccd route 10.0.0.0 255.255.255.0 ;learn-address ./script ;push "redirect-gateway def1 bypass-dhcp" ;push "dhcp-option DNS 208.67.220.220" client-to-client keepalive 10 120 ;tls-auth /etc/openvpn/keys/ta.key 0 # This file is secret ;cipher BF-CBC # Blowfish (default) ;cipher AES-128-CBC # AES ;cipher DES-EDE3-CBC # Triple-DES comp-lzo ;max-clients 100 user nobody group nobody persist-key persist-tun status openvpn-status.log ;log openvpn.log ;log-append openvpn.log verb 3 ;mute 20[/code] Here is the ccd file for the client: [/code] iroute 10.0.0.0 255.255.255.0 [/code] The docs go on to say: [quote] Next, you must set up a route on the server-side LAN gateway to route the VPN client subnet (10.8.0.0/24) to the OpenVPN server (this is only necessary if the OpenVPN server and the LAN gateway are different machines).[/quote] I'm not sure I did this right. The gateway for a GCE slice is the first ip on the subnet, which in this case is 10.250.0.1. In the network settings for the 10.250.0.0/16 (in the GCS web front-end) I added a route for both the 10.254.254.0/24 and 10.0.0.0/24 subnets, and made the "next hop" point to the OpenVPN server (presumably ip address 10.250.0.2). In addition, in the firewall settings for the subnet I made sure port 1194 was open for any source. When I start the server I see this: [code] Fri Mar 18 05:31:31 2016 OpenVPN 2.3.10 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Jan 4 2016 Fri Mar 18 05:31:31 2016 library versions: OpenSSL 1.0.1e-fips 11 Feb 2013, LZO 2.06 Fri Mar 18 05:31:31 2016 Diffie-Hellman initialized with 2048 bit key Fri Mar 18 05:31:31 2016 Socket Buffers: R=[212992->212992] S=[212992->212992] Fri Mar 18 05:31:31 2016 ROUTE_GATEWAY 10.250.0.1 Fri Mar 18 05:31:31 2016 TUN/TAP device tun0 opened Fri Mar 18 05:31:31 2016 TUN/TAP TX queue length set to 100 Fri Mar 18 05:31:31 2016 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0 Fri Mar 18 05:31:31 2016 /usr/sbin/ip link set dev tun0 up mtu 1500 Fri Mar 18 05:31:31 2016 /usr/sbin/ip addr add dev tun0 local 10.254.254.1 peer 10.254.254.2 Fri Mar 18 05:31:31 2016 /usr/sbin/ip route add 10.0.0.0/24 via 10.254.254.2 Fri Mar 18 05:31:31 2016 /usr/sbin/ip route add 10.254.254.0/24 via 10.254.254.2 Fri Mar 18 05:31:31 2016 GID set to nobody Fri Mar 18 05:31:31 2016 UID set to nobody Fri Mar 18 05:31:31 2016 UDPv4 link local (bound): [undef] Fri Mar 18 05:31:31 2016 UDPv4 link remote: [undef] Fri Mar 18 05:31:31 2016 MULTI: multi_init called, r=256 v=256 Fri Mar 18 05:31:31 2016 IFCONFIG POOL: base=10.254.254.4 size=62, ipv6=0 Fri Mar 18 05:31:31 2016 ifconfig_pool_read(), in='ads-120elm-pfsense,10.250.0.3', TODO: IPv6 Fri Mar 18 05:31:31 2016 succeeded -> ifconfig_pool_set() Fri Mar 18 05:31:31 2016 IFCONFIG POOL LIST Fri Mar 18 05:31:31 2016 Initialization Sequence Completed[/code] I don't know where that 10.254.254.2 address is coming from. I am able to connect from the pfsense client, but I cannot ping the ip addresses of either the tun or eth interfaces from a computer on the client lan, nor can I ping any computer on the client lan from the server side, or the gateway, or the 10.254.254.4/5/6 address I see mentioned in the server's console output when the cllient connects. [code] Fri Mar 18 05:42:27 2016 69.115.109.60:45723 TLS: Initial packet from [AF_INET]69.115.109.62:45723, sid=34f4556a 9aae74a9 Fri Mar 18 05:42:30 2016 69.115.109.60:45723 VERIFY OK: depth=1, C=US, ST=CA, L=Newark, O=ADS, OU=MyOrganizationalUnit, CN=ads-vpn-server-1, name=EasyRSA, emailAddress=me@mymail.com Fri Mar 18 05:42:30 2016 69.115.109.62:45723 VERIFY OK: depth=0, C=US, ST=CA, L=Newark, O=ADS, OU=MyOrganizationalUnit, CN=ads-pfsense, name=EasyRSA, emailAddress=me@mymail.com Fri Mar 18 05:42:31 2016 69.115.109.62:45723 WARNING: 'dev-type' is used inconsistently, local='dev-type tun', remote='dev-type tap' Fri Mar 18 05:42:31 2016 69.115.109.62:45723 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1542', remote='link-mtu 1574' Fri Mar 18 05:42:31 2016 69.115.109.62:45723 WARNING: 'tun-mtu' is used inconsistently, local='tun-mtu 1500', remote='tun-mtu 1532' Fri Mar 18 05:42:31 2016 69.115.109.62:45723 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key Fri Mar 18 05:42:31 2016 69.115.109.62:45723 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Fri Mar 18 05:42:31 2016 69.115.109.62:45723 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key Fri Mar 18 05:42:31 2016 69.115.109.62:45723 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication Fri Mar 18 05:42:31 2016 69.115.109.62:45723 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA Fri Mar 18 05:42:31 2016 69.115.109.62:45723 [ads-120elm-pfsense] Peer Connection Initiated with [AF_INET]69.115.109.62:45723 Fri Mar 18 05:42:31 2016 ads-120elm-pfsense/69.115.109.62:45723 MULTI_sva: pool returned IPv4=10.254.254.6, IPv6=(Not enabled) Fri Mar 18 05:42:31 2016 ads-120elm-pfsense/69.115.109.62:45723 MULTI: Learn: 10.254.254.6 -> ads-120elm-pfsense/69.115.109.60:45723 Fri Mar 18 05:42:31 2016 ads-120elm-pfsense/69.115.109.62:45723 MULTI: primary virtual IP for ads-120elm-pfsense/69.115.109.60:45723: 10.254.254.6 Fri Mar 18 05:42:33 2016 ads-120elm-pfsense/69.115.109.62:45723 PUSH: Received control message: 'PUSH_REQUEST' Fri Mar 18 05:42:33 2016 ads-120elm-pfsense/69.115.109.62:45723 send_push_reply(): safe_cap=940 Fri Mar 18 05:42:33 2016 ads-120elm-pfsense/69.115.109.62:45723 SENT CONTROL [ads-120elm-pfsense]: 'PUSH_REPLY,route 10.250.0.0 255.255.0.0,route 10.254.254.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.254.254.6 10.25 4.254.5' (status=1)[/code] I'm only able to ping 10.254.254.6, and only from the client side. The settings on the client side are the same as in the image in my last post: [img]http://snag.gy/ums2C.jpg[/img] I've tried with and without the remote networks set explicitly on the client side. I've tried tcp instead of udp. I've tried disabling the firewall and selinux. I've tried the tap0 interface and server-bridge method. I was able to get the bridge-start script working by adding the default gateway at the end of the script. Unfortunately, though the docs said that computers on the server lan would be visible out of the box, they were not. As far as making the client-side subnet available to the server-side, I do not know enough about pfsense to follow that part of the docs (which is not descriptive). Any help is much appreciated.

[quote="FuriousGeorge"]Re: Site-to-site OpenVPN Server on GCE: Searching for good (never mind...)[/quote]what does this mean ? [quote="FuriousGeorge"]I don't know where that 10.254.254.2 address is coming from[/quote]This route is how OpenVPN handles --[b]topology net30[/b] --[b]topology subnet[/b] is probably easier to configure .. See [b]--topology[/b] in [url=https://community.openvpn.net/openvpn/wiki/Openvpn23ManPage]The Manual v23x[/url]

If you're able to establish link between two point, then it is a routing and firewall problem. However, the most difficult problem I had was to find what and where the problem is. I suggest you to check each interfaces of both end with this command while ping to each other's internal subnets. This can be used on pfSense: tcpdump -n -i 'icmp' I don't know what command does the same thing for Linux but I think you can search it from the Internet. With this command, you will be able to know your ping command packet reaches the interface you're listening. If there is another computer in the client subnet with 10.0.0.2, for example, from 10.0.0.2 you may ping 10.250.0.2 If every config is done correctly, the ping request packet will be transported like this: 10.0.0.2 -> 10.0.0.1 -> 10.254.254.6(This may be skipped) -> (OpenVPN internal gateways, which may include multiple IPs with 10.254.254.xxx) -> 10.254.254.1(Server's Tun IP, also can be skipped) -> 10.250.0.2(Server's internal subnet IP) From 10.0.0.2 to 10.250.0.2, if every routes is set correctly then you will be able to see the ping request from 10.250.0.2's interface. If even requests are not seen from 10.250.0.2's interface via the command, then you will need to check every intermediate IPs to see if there's any problem. With the command above, you can know where the problem is. If the route is set correctly and the requests arrive to 10.250.0.2, but if there is no reply to 10.0.0.2, the server, then it means it is a firewall problem. Firewall will have to be notified of subnets linked via OpenVPN. The firewalls will usually not accept any packets from unknown subnets. Of course, you can use tcpdump command to know where the flow is blocked. For example, without any settings, the Server's firewall will know 10.250.0.0/16 and 10.254.254.0/24, but it doesn't know about 10.0.0.0/24. So, even when reply packets from 10.0.0.2 arrives, the firewall will block the packet. And in this case, of course, you will see no response from 10.0.0.2. So we need to set a rule to pass packets from the source 10.0.0.0/24 to the server for the interface of 10.250.0.2 belongs.

[quote="Traffic"][quote="FuriousGeorge"]Re: Site-to-site OpenVPN Server on GCE: Searching for good (never mind...)[/quote]what does this mean ?[/quote] Sorry for mix up. I posted something then realized it wasn't relevant. That's all it means.

Site-to-site OpenVPN Server on GCE: Searching for good docs

FuriousGeorge

cr2025x1 wroteThe steps' questions are only valid when given condition is fulfilled.
And that's weird. You were able to make a site-to-site even when you used NAT on both sides? That is very weird.
You're talking about NATing packets outbound to WAN, right? If you're NATing packets going to VPN connection, well... It is just not possible as I know.
Well, however, if NAT is completely disabled then DHCP clients of the pfSense will not be able to reach the Internet, except the router itself.

I'm not sure why you'd think that. There are docs for setting up a site-to-site VPN with pfsense on both sides with NAT. https://doc.pfsense.org/index.php/OpenVPN_Site_To_Site

Am I not understanding you right?

cr2025x1 wrote Then I suggest another thing.

You said you can't ping to 10.0.0.1 from the server. However, the pfSense can ping the server side.
And you also said you're allowing every inbound packets. Are you saying about rules of WAN interface?
To me, it seems obvious that this is a firewall problem.
If that my guess is right, you should add the rule to LAN interface, not WAN interface, considering my experience.
And I will guess that the pfSense's 10.0.0.0/24 resides in its LAN interface, which I believe 'rl0'.
Because if I omit the whole process related to the OpenVPN tunneling, the ping request from server will first come to OpenVPN interface ovpnc1, not WAN. It will completely bypass WAN zone.
And the request packet will try to enter LAN zone, as the recipient of the packet is 10.0.0.1 and the interface with that address is in LAN zone.
But if the firewall has no rules at LAN zone to pass packets that the source of packet is 10.250.0.0/16, it will block it. I said LAN, not WAN.

https://www.dropbox.com/s/7zvsg6dpjke7y43/Screenshot_2016-03-20-13-57-26.jpg?dl=0
https://www.dropbox.com/s/67qcsa9j1u8kzxm/Screenshot_2016-03-20-14-00-03.jpg?dl=0

I tried it, even though I had a strong feeling it would not work. It may be needed at some point, but right now I can't even see pings reaching pfsense anywhere. I already checked the ovpnc1 interface for the pings from the server, and found nothing.

FuriousGeorge

Traffic wroteAs a suggestion .. it is much easier to help if you limit one thread to one problem instead of this monolithic approach.
You are welcome to start more than one thread ..

There are so few people who try to help and so many who want help .. :geek:

Per request, this thread has moved here.

Traffic

Not a request but a suggestion:

FuriousGeorge wrotelimit one thread to one problem

Now reading your new thread 😉

« Previous Page