Site to site VPN can ping but no connection to services

Ask questions about your Access Server configuration here.
Post Reply
graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Site to site VPN can ping but no connection to services

Post by graciejane » Sun Jul 26, 2020 1:43 am

Hi,

Looking for some expert help here please. Sorry if this has already been answered but I have spent hours googling and although there are quite a few results that seem to be my issue on the surface, the details aren't the same. I did find something that suggested the MTU might be to blame, but I haven't adjusted that from the defaults and I suspect there is a more fundamental explanation.

I set up my first site to site VPN using OpenVPN and was excited when I could ping hosts on the other side. It didn't last long as I discovered that nothing other than ping worked. Things like RDP, SSH and HTTP. These things do work locally host-to-host on the remote network so I know they are not refusing connections. Also I can access them via a SSH tunnel / port forward so it's all pointing to something being wrong with my VPN config.

The open access server in my case is at the remote location (remote from my perspective) but to avoid confusion I'll refer to the server side as local and the client side (my house) as remote.

The setup:

The entire local network including the access server is virtual, hosted on Proxmox Virtual Environment (PVE).
There is one public IP on the PVE host and all the VMs are on a private subnet. Access to the internet is via NAT. Incoming connections via port forwarding from the PVE host. I used a pre-built access server VM downloaded from OpenVPN. The remote side is a typical home network with a private local net behind NAT, only in my case there's an extra layer of NAT because I'm using a pfSense router and my ISPs router can't be put in bridge mode. It doesn't seem to cause any issues (unless it's the cause of this one). The OpenVPN client is on a CentOS 8 server in my home on the private network. Routing is done on PVE and pfSense.

I followed this guide to set it all up:
https://openvpn.net/vpn-server-resource ... te-to-site
Something I found puzzling with that "how-to" is that it never mentioned the WAN IP address of the router where the access server is. How can the client connect if it doesn't know where in the internet the server is? Anyway I got that sorted.

Long story short I have it successfully connecting. The client server can ping the access server and vice versa. Hosts on the remote network (client side) can ping the access server and any host on the local network (server side). Hosts on the local network can ping the client server, but not hosts on the client network. I'm not sure if this is related to the problem or if it's a separate issue.

Local site private subnet is 10.0.1.0/24
Local site gateway/router is 10.0.1.1
Local site OpenVPN access server is 10.0.1.5
Remote site private subnet is 10.0.2.0/24
Remote site gateway/router is 10.0.2.1
Remote site OpenVPN client "server" is 10.0.2.5
VPN tunnel subnet is 172.27.224.0/20, automatically generated by the access server during setup.

traceroute to host on local net from host on remote net (looks good):

Code: Select all

C:\Users\grace>tracert -d 10.0.1.20

Tracing route to 10.0.1.20 over a maximum of 30 hops

  1     1 ms     2 ms    <1 ms  10.0.2.1
  2     1 ms     1 ms     1 ms  10.0.2.5
  3   340 ms   317 ms   316 ms  172.27.232.1
  4   319 ms   316 ms   316 ms  10.0.1.20

Trace complete.
traceroute to host on remote net from host on local net (this may the problem, ping also fails):

Code: Select all

[grace@web1 ~]$ traceroute -n 10.0.2.101
traceroute to 10.0.2.101 (10.0.2.101), 30 hops max, 60 byte packets
 1  10.0.1.5  0.768 ms  0.714 ms  1.513 ms
 2  172.27.232.4  320.923 ms  320.950 ms  321.052 ms
 3  172.27.232.4  321.181 ms !X  321.503 ms !X  321.580 ms !X
[grace@web1 ~]$
Static routes on pfSense (remote site):

Code: Select all

Network		Gateway
10.0.1.0/24	10.0.2.5
172.27.224.0/24	10.0.2.5
Relevant static routes and port forwards on PVE (local site):

Code: Select all

# Routes required for OpenVPN server vpn2
post-up ip route add 172.27.224.0/20 via 10.0.1.5
pre-down ip route del 172.27.224.0/20 via 10.0.1.5
post-up ip route add 10.0.2.0/24 via 10.0.1.5
pre-down ip route del 10.0.2.0/24 via 10.0.1.5

# Port TCP 9443(443) and UDP 9194(1194) to vpn2
post-up iptables -t nat -A PREROUTING -i vmbr0 -p tcp --dport 9443 -j DNAT --to 10.0.1.5:443
post-down iptables -t nat -D PREROUTING -i vmbr0 -p tcp --dport 9443 -j DNAT --to 10.0.1.5:443
post-up iptables -t nat -A PREROUTING -i vmbr0 -p udp --dport 9194 -j DNAT --to 10.0.1.5:1194
post-down iptables -t nat -D PREROUTING -i vmbr0 -p udp --dport 9194 -j DNAT --to 10.0.1.5:1194
Any suggestions how to start troubleshooting this?

Thanks!

graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Sun Jul 26, 2020 11:45 am

Client config (sensitive info removed - public IP and crypto certs/keys):
Client Config
# Automatically generated OpenVPN client config file
# Generated on Sat Jul 25 18:09:38 2020 by openvpnas2

# Default Cipher
cipher AES-256-CBC
# Note: this config file contains inline private keys
# and therefore should be kept confidential!
# Note: this configuration is user-locked to the username below
# OVPN_ACCESS_SERVER_USERNAME=stsvpn
# Define the profile name of this particular configuration file
# OVPN_ACCESS_SERVER_PROFILE=stsvpn@10.0.1.5/AUTOLOGIN
# OVPN_ACCESS_SERVER_AUTOLOGIN=1
# OVPN_ACCESS_SERVER_CLI_PREF_ALLOW_WEB_IMPORT=True
# OVPN_ACCESS_SERVER_CLI_PREF_BASIC_CLIENT=False
# OVPN_ACCESS_SERVER_CLI_PREF_ENABLE_CONNECT=False
# OVPN_ACCESS_SERVER_CLI_PREF_ENABLE_XD_PROXY=True
# OVPN_ACCESS_SERVER_WSHOST=10.0.1.5:443
# OVPN_ACCESS_SERVER_WEB_CA_BUNDLE_START
# -----BEGIN CERTIFICATE-----
# -----END CERTIFICATE-----
# OVPN_ACCESS_SERVER_WEB_CA_BUNDLE_STOP
# OVPN_ACCESS_SERVER_IS_OPENVPN_WEB_CA=1
setenv FORWARD_COMPATIBLE 1
client
server-poll-timeout 4
nobind
remote x.x.x.x 9194 udp
remote x.x.x.x 9443 tcp
#remote 10.0.1.5 1194 udp
#remote 10.0.1.5 443 tcp
#remote 10.0.1.5 1194 udp
#remote 10.0.1.5 1194 udp
#remote 10.0.1.5 1194 udp
#remote 10.0.1.5 1194 udp
#remote 10.0.1.5 1194 udp
dev tun
dev-type tun
ns-cert-type server
setenv opt tls-version-min 1.0 or-highest
reneg-sec 604800
sndbuf 0
rcvbuf 0
# NOTE: LZO commands are pushed by the Access Server at connect time.
# NOTE: The below line doesn't disable LZO.
comp-lzo no
verb 3
setenv PUSH_PEER_INFO

<ca>
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
</ca>

<cert>
-----BEGIN CERTIFICATE-----
-----END CERTIFICATE-----
</cert>

<key>
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----
</key>

key-direction 1
<tls-auth>
#
# 2048 bit OpenVPN static key (Server Agent)
#
-----BEGIN OpenVPN Static key V1-----
-----END OpenVPN Static key V1-----
</tls-auth>

## -----BEGIN RSA SIGNATURE-----
## DIGEST:sha256
## -----END RSA SIGNATURE-----
## -----BEGIN CERTIFICATE-----
## -----END CERTIFICATE-----
## -----BEGIN CERTIFICATE-----
## -----END CERTIFICATE-----


Not sure where to find the server config. There's no /etc/openvpn directory.

/usr/local/openvpn_as/etc/as.conf:
Server as.conf
# OpenVPN AS 1.1 configuration file
#
# NOTE: The ~ symbol used below expands to the directory that
# the configuration file is saved in

# remove for production
# DEBUG=false

# enable AS Connect functionality
AS_CONNECT=true

# temporary directory
tmp_dir=~/tmp

lic.dir=~/licenses

# run_start retries
run_start_retry.give_up=60
run_start_retry.resample=10

# enable client gateway
sa.show_c2s_routes=true

# certificates database
certs_db=sqlite:///~/db/certs.db

# user properties DB
user_prop_db=sqlite:///~/db/userprop.db

# configuration DB
config_db=sqlite:///~/db/config.db

# configuration DB Local
config_db_local=sqlite:///~/db/config_local.db

# cluster DB
cluster_db=sqlite:///~/db/cluster.db

# notification DB
notification_db=sqlite:///~/db/notification.db

# log DB
log_db=sqlite:///~/db/log.db

# wait this many seconds between failed retries
db_retry.interval=1

# how many retries to attempt before failing
db_retry.n_attempts=6

# On startup, wait up to n seconds for DB files to become
# available if they do not yet exist. This is generally
# only useful on secondary nodes used for standby purposes.
# db_startup_wait=

# Node type: PRIMARY|SECONDARY. Defaults to PRIMARY.
# node_type=

# bootstrap authentication via PAM -- allows
# admin to log into web UI before authentication
# system has been configured. Configure PAM users
# allowed to access via the bootstrap auth mechanism.
boot_pam_service=openvpnas
boot_pam_users.0=openvpn
# boot_pam_users.1=
# boot_pam_users.2=
# boot_pam_users.3=
# boot_pam_users.4=

# System users that are allowed to access the server agent XML API.
# The user that the web server will run as should be in this list.
system_users_local.0=root
system_users_local.1=openvpn_as

# The user/group that the web server will run as
cs.user=openvpn_as
cs.group=openvpn_as

# socket directory
general.sock_dir=~/sock

# path to linux openvpn executable
# if undefined, find openvpn on the PATH
#general.openvpn_exe_path=

# source directory for OpenVPN Windows executable
# (Must have been built with MultiFileExtract)
sa.win_exe_dir=~/exe

# The company name will be shown in the UI
# sa.company_name=Access Server

# server agent socket
sa.sock=~/sock/sagent

# If enabled, automatically generate a client configuration
# when a client logs into the site and successfully authenticates
cs.auto_generate=true

# files for web server (PEM format)
cs.ca_bundle=~/web-ssl/ca.crt
cs.priv_key=~/web-ssl/server.key
cs.cert=~/web-ssl/server.crt

# web server will use three consecutive ports starting at this
# address, for use with the OpenVPN port share feature
cs.dynamic_port_base=870

# which service groups should be started during
# server agent initialization
sa.initial_run_groups.0=web_group
#sa.initial_run_groups.1=openvpn_group

# use this twisted reactor
sa.reactor=epoll

# The unit number of this particular AS configuration.
# Normally set to 0. If you have multiple, independent AS instances
# running on the same machine, each should have a unique unit number.
sa.unit=0

# If true, open up web ports on the firewall using iptables
iptables.web=true

vpn.server.user=openvpn_as
vpn.server.group=openvpn_as



/usr/local/openvpn_as/etc/config.json:
Server config.json
{
"Default": {
"auth.ldap.0.name": "My LDAP servers",
"auth.ldap.0.ssl_verify": "internal",
"auth.ldap.0.timeout": "4",
"auth.ldap.0.use_ssl": "never",
"auth.module.type": "local",
"auth.pam.0.service": "openvpnas",
"auth.radius.0.acct_enable": "false",
"auth.radius.0.name": "My Radius servers",
"cs.cws_proto_v2": "true",
"cs.prof_sign_web": "true",
"cs.ssl_method": "SSLv3",
"cs.tls_version_min": "1.1",
"sa.initial_run_groups.0": "web_group",
"sa.initial_run_groups.1": "openvpn_group",
"vpn.client.basic": "false",
"vpn.client.cipher": "AES-256-CBC",
"vpn.general.osi_layer": "3",
"vpn.server.cipher": "AES-256-CBC",
"vpn.server.group_pool.0": "172.27.240.0/20",
"vpn.server.tls_auth": "true",
"vpn.server.tls_version_min": "1.2",
"vpn.tls_refresh.do_reauth": "true",
"vpn.tls_refresh.interval": "360"
},
"_INTERNAL": {
"run_api.active_profile": "Default",
"webui.edit_profile": "Default"
}
}

graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Mon Jul 27, 2020 3:42 am

Me again... just curious as to why this was moved to enterprise business solutions? It's only for my personal use. I get the feeling I'm using the wrong product, but there's no mention of anything else on the website.

Thanks.

User avatar
TinCanTech
OpenVPN Protagonist
Posts: 7584
Joined: Fri Jun 03, 2016 1:17 pm

Re: Site to site VPN can ping but no connection to services

Post by TinCanTech » Mon Jul 27, 2020 11:04 am


graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Mon Jul 27, 2020 11:53 am

Thank you :)

Unfortunately I have bigger fish to fry right now. I seem to have completely broken port forwarding on the public IP which means my website is down :(

User avatar
Pippin
Forum Team
Posts: 831
Joined: Wed Jul 01, 2015 8:03 am

Re: Site to site VPN can ping but no connection to services

Post by Pippin » Mon Jul 27, 2020 12:34 pm

What you have installed is the OpenVPN Access Server which is the commercial product.
You can use it up to two simultaneous connections, see here under "Is there a trial version available?":
https://openvpn.net/vpn-server-resource ... licensing/

Then there is the OpenVPN Community Version which does not have a GUI:
https://openvpn.net/community/
https://community.openvpn.net/openvpn/wiki/HOWTO

graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Tue Jul 28, 2020 7:39 am

Thanks, yeah I didn't realize at the time that the client software it was referring to could also act as the server. I am now enlightened :) I built a new server from scratch using the community version with new config files for both server and client. Same result - connects OK, ping is successful, nothing else works. This time I can ping from any machine on either private subnet to any machine on the other private subnet, so that's an improvement. Although ping gets through, it seems that other traffic is being dropped somewhere. My mission is to figure out where and why.

Is using tools like wireshark and tcpdump my best bet at this point?

User avatar
TinCanTech
OpenVPN Protagonist
Posts: 7584
Joined: Fri Jun 03, 2016 1:17 pm

Re: Site to site VPN can ping but no connection to services

Post by TinCanTech » Tue Jul 28, 2020 10:37 am

Sounds suspiciously like local firewalls ..

graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Tue Jul 28, 2020 2:47 pm

Bingo. Completely disabling the firewall on both OpenVPN client and server enables it to work. Apologies for wasting your time. I didn't think it would be that if I could ping end to end through the tunnel. We live and learn!

So, given that these machines are not directly exposed to the internet and I'm the only person accessing anything on these networks (except for a web server only on port 80 and 443 via port forwarding), is it safe to leave it like that?

User avatar
TinCanTech
OpenVPN Protagonist
Posts: 7584
Joined: Fri Jun 03, 2016 1:17 pm

Re: Site to site VPN can ping but no connection to services

Post by TinCanTech » Tue Jul 28, 2020 3:16 pm

graciejane wrote:
Tue Jul 28, 2020 2:47 pm
is it safe
That is your decision .. you could learn how to configure the firewalls correctly.

graciejane
OpenVpn Newbie
Posts: 7
Joined: Sun Jul 26, 2020 12:40 am

Re: Site to site VPN can ping but no connection to services

Post by graciejane » Wed Jul 29, 2020 7:13 am

TinCanTech wrote:
Tue Jul 28, 2020 3:16 pm
That is your decision
No shit
TinCanTech wrote:
Tue Jul 28, 2020 3:16 pm
.. you could learn how to configure the firewalls correctly.
You could learn how to be less condescending.

User avatar
TinCanTech
OpenVPN Protagonist
Posts: 7584
Joined: Fri Jun 03, 2016 1:17 pm

Re: Site to site VPN can ping but no connection to services

Post by TinCanTech » Wed Jul 29, 2020 11:13 am

graciejane wrote:
Wed Jul 29, 2020 7:13 am
TinCanTech wrote:
Tue Jul 28, 2020 3:16 pm
That is your decision
No shit
TinCanTech wrote:
Tue Jul 28, 2020 3:16 pm
.. you could learn how to configure the firewalls correctly.
You could learn how to be less condescending.
You asked -- I Answered.

If you can't take the heat then stay out of the kitchen...

Post Reply