[Solved] Issue getting to LAN behind VPN Server

Samples of working configurations.

Moderators: TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech, TinCanTech

Locked
jeffboyce
OpenVpn Newbie
Posts: 3
Joined: Wed Nov 25, 2015 6:34 pm

[Solved] Issue getting to LAN behind VPN Server

Post by jeffboyce » Wed Dec 02, 2015 11:55 pm

I had initially posted a request for assistance on the mailing list and have not resolved my issue. It was recommended that I move my issue here to the forum for better assistance. Since my request is new to this forum I will provide a complete description of my issue and what steps I have taken at this point (so the post may get a little long with detail).

MY GOAL
To be able to directly access additional boxes behind my OpenVPN server from a remote client. The boxes may be either other servers, or desktop systems on the same network.

MY NETWORK TOPOLOGY
Server LAN network 192.168.112.0/24
VPN Server LAN IP 192.168.112.50
VPN Server IP 10.9.8.1

DNS/DHCP Server 192.168.112.51
Gateway/Firewall/Router 192.168.112.11
* Note that my network router is on a different box than my VPN Server *

Remote client network 192.168.123.0/24
VPN Client IP 10.9.8.x

I have a basic VPN connection established. I can connect from a remote (Windows) client to my OpenVPN Server (CentOS 6.7) without any issue. The client and server configs are posted below. I have not included the connection log from the client. Since I have an established connection I don’t expect that to be of any value in resolving this issue, but please let me know otherwise.
  • CLIENT CONFIG
    client
    dev tun
    proto udp
    remote xxx.yy.190.20 1195
    pull
    nobind
    persist-key
    persist-tun
    tls-client
    ca "C:\\Program Files\\OpenVPN\\config\\yca.crt"
    cert "C:\\Program Files\\OpenVPN\\config\\JABdougfirhomeclient.crt"
    key "C:\\Program Files\\OpenVPN\\config\\JABdougfirhomeclient.key"
    ns-cert-type server
    resolv-retry infinite
    comp-lzo
    route-method exe
    route-delay 2
    verb 4
  • SERVER CONFIG
    local 192.168.112.50
    port 1195
    proto udp
    dev tun
    tls-server
    ca /etc/openvpn/easy-rsa/2.0/keys/ca.crt
    cert /etc/openvpn/easy-rsa/2.0/keys/SequoiaVPNserver.crt
    key /etc/openvpn/easy-rsa/2.0/keys/SequoiaVPNserver.key
    dh /etc/openvpn/easy-rsa/2.0/keys/dh2048.pem
    server 10.9.8.0 255.255.255.0
    float
    ifconfig-pool-persist /etc/openvpn/ipp.txt 120
    push "route 192.168.112.0 255.255.255.0"
    push "explicit-exit-notify 3"
    keepalive 10 120
    comp-lzo
    user nobody
    group nobody
    persist-key
    persist-tun
    status /var/log/openvpn-status.log
    log-append /var/log/openvpn.log
    verb 4
MY EDUCATION AND RESEARCH
I have read through several how-to’s regarding accessing LAN resources behind the VPN server, including one by Mimiko, one by Krzee, and the simple description on the OpenVPN.net HowTo. From that information I have reviewed, I recognize that there are four different approaches to solving this issue.
  • 1. Move the OpenVPN server to the LAN gateway.
    Not feasible on my company network.

    2. Add a static route for remote network to *every host’s* routing table.
    Not a flexible or scalable solution that I would want to implement or maintain over the long term.

    3. All packets from the remote network and tunnel itself are masqueraded to represent the OpenVPN host local IP, therefore all packets would be returned/routed back to that IP.
    This has the disadvantage of obscuring the source of each VPN connection on my LAN, which may cause problems with future network diagnostics. Therefore I would like to avoid this approach.

    4. Add route to the LAN default gateway.
    The seems like the most efficient solution to what I believe to be a routing issue, and is the approach I would like to implement.
WHAT I HAVE DONE
I have ipforwarding enabled on the VPN server (CentOS 6.7) box. I can access the VPN server at ...50, and I would like to access another server which is at ...53. Results from pinging the VPN server, and another server on the LAN behind the VPN server are shown below. Any box behind the VPN server returns a response of Destination Host Unreachable.

c:\>ping 192.168.112.50
Pinging 192.168.112.50 with 32 bytes of data:
Reply from 192.168.112.50: bytes=32 time=75ms TTL=64

c:\>ping 192.168.112.53
Pinging 192.168.112.53 with 32 bytes of data:
Reply from 10.9.8.1: Destination host unreachable.

Researching the destination host unreachable response led me to the following good description of how this message is generated via network protocols.
* This message indicates one of two problems: either the local system has no route to the desired destination, or a remote router reports that it has no route to the destination. The two problems can be distinguished by the form of the message. If the message is simply "Destination Host Unreachable," then there is no route from the local system, and the packets to be sent were never put on the wire. Use the Route utility to check the local routing table. If the message is "Reply From < IP address >: Destination Host Unreachable," then the routing problem occurred at a remote router, whose address is indicated by the "< IP address >" field. Use the appropriate utility or facility to check the IP routing table of the router assigned the IP address of < IP address >. *
Source: https://technet.microsoft.com/en-us/lib ... 40095.aspx

Based on the ping response, and since *my LAN router is not the same box as the VPN server*, I had previously concluded that I needed to add a route on my LAN default gateway for the VPN network IP subnet pointing to the OpenVPN box (following approach #4 described above). I added a static route on my default gateway (a PfSense box) to redirect any VPN network traffic back to the VPN box, which I believe should send it back through the tunnel to the point of origin. However, this did not resolve my issue, so I must be mistaken about my routing somewhere, or there is something else that I have not considered.

CURRENT ROUTING AND FIREWALL INFO
The current routing table for the gateway/router (PfSense) box shows:

Code: Select all

[2.1.2-RELEASE][admin@pfgateway.mei.lan]/root(1): netstat -rn
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            xxx.yy.190.17      UGS         1 663197177   bge0
10.9.8.0/24        192.168.112.50     UGS         0        0    rl0
xxx.yy.190.16/28   link#1             U           0 49927480   bge0
xxx.yy.190.18      link#1             UHS         0        0    lo0
xxx.yy.190.19      link#1             UHS         0        0    lo0 =>
xxx.yy.190.19/32   link#1             U           0        0   bge0
xxx.yy.190.20      link#1             UHS         0        0    lo0 =>
xxx.yy.190.20/32   link#1             U           0        0   bge0
xxx.yy.190.21      link#1             UHS         0        0    lo0 =>
xxx.yy.190.21/32   link#1             U           0        0   bge0
xxx.yy.190.22      link#1             UHS         0        0    lo0 =>
xxx.yy.190.22/32   link#1             U           0        0   bge0
127.0.0.1          link#6             UH          0      560    lo0
192.168.112.0/24   link#2             U           0 848109484    rl0
192.168.112.11     link#2             UHS         0 99857580    lo0
The network and routing information for the VPN box shows:

Code: Select all

[root@sequoia ~]# ifconfig
eth0      Link encap:Ethernet  HWaddr aa:bb:00:9D:B2:49
          inet addr:192.168.112.50  Bcast:192.168.112.255 Mask:255.255.255.0
          inet6 addr: fe80::5054:ff:fe9d:b249/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:188054402 errors:0 dropped:0 overruns:0 frame:0
          TX packets:169321935 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:36069946217 (33.5 GiB)  TX bytes:47188246310 (43.9 GiB)

< snip lo >

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet addr:10.9.8.1  P-t-P:10.9.8.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:9066254 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7591995 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1029571120 (981.8 MiB)  TX bytes:1210868888 (1.1 GiB)

[root@192.168.112.50 ~]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.9.8.2        *               255.255.255.255 UH    0 0        0 tun0
192.168.112.0   *               255.255.255.0   U     0 0        0 eth0
10.9.8.0        10.9.8.2        255.255.255.0   UG    0 0        0 tun0
link-local      *               255.255.0.0     U     1002 0        0 eth0
default         pfgateway.mei.l 0.0.0.0         UG    0 0        0 eth0
The above routing information on both the gateway/router and VPN boxes appear to be correct to me, but I am still in this pickle so I could very well be wrong.

The next idea that I have considered is that the firewall on the VPN server box might be restricting the packets. Below are the firewall rules for the VPN server. The only possibility that I was thinking here was that I might need to add an accept rule for the 10.9.8.0/24 network in the forward chain. But I thought that by having ipforwarding turn on it wasn’t needed here, so I haven’t made any changes.

Code: Select all

[root@sequoia ~]# iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 289M   50G ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
  573 22456 ACCEPT     icmp --  *      *       0.0.0.0/0            0.0.0.0/0
    1    60 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
  124  6504 ACCEPT     all  --  tun0   *       0.0.0.0/0            0.0.0.0/0
 261K   38M ACCEPT     all  --  *      *       192.168.112.0/24     0.0.0.0/0
    0     0 ACCEPT     all  --  *      *       10.9.8.0/24          0.0.0.0/0
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:1194
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:137
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:138
   62  2988 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:139
  267 14588 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:445 state NEW
 3276  177K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:22
   65  3504 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:10000
   47  1974 ACCEPT     udp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW udp dpt:1195
 117K 5194K REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 108K   12M REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT 261M packets, 62G bytes)
 pkts bytes target     prot opt in     out     source               destination
As I mentioned previously I would like to resolve this issue following approach #4 described above which should result in a simple routing solution. But I am a little uncertain about what specific routing change would be necessary. If there is any additional information needed to assist with diagnosing this issue that I have overlooked, please let me know. While I am ultimately looking for a solution, I am not looking for a handout. I would like some guidance on how to diagnose this further (if any additional diagnosis is needed), then education on how to implement a resolution. Thanks.

Jeff

jeffboyce
OpenVpn Newbie
Posts: 3
Joined: Wed Nov 25, 2015 6:34 pm

Re: Issue getting to LAN behind VPN Server

Post by jeffboyce » Tue Dec 29, 2015 7:36 pm

< bump >
Since this was held so long by the moderators that it originally posted 4 pages down the list.

jeffboyce
OpenVpn Newbie
Posts: 3
Joined: Wed Nov 25, 2015 6:34 pm

[Solved] Issue getting to LAN behind VPN Server

Post by jeffboyce » Wed Jan 13, 2016 8:35 pm

I have had help from some knowledgeable people on the OpenVPN mailing list that gave me additional diagnostic tests to consider to pinpoint the root cause of my issue. I will give a summary of those additional suggestions, and what I did to solve this issue for the benefit of anyone else finding this topic in their search for a similar issue/solution.

Additional diagnostic tests to consider; listed by author.
Selva
The configs and routes on server & router look fine, but not clear whether the windows box (client) has correctly set up the pushed route to the 192.168.112.0/24 subnet. Please check the routing table on the windows client.

Gert
Run wireshark/tcpdump on the tun interfaces, see whether packets are going *in* the tunnel (= routing on the client is right) and whether reply packets are coming back in (= routing on the other end is right).

Optionally run wireshark on the remote machine that isn't reachable and see whether packets coming from the server tun actually arrive there, or are eaten by a firewall rule in between, etc.

Morten
What happens when you try a traceroute from the other server .53 to the vpn-servers tunnel ip 10.9.8.1 ?

debbie10t
have you tried (Method 2) adding a route to **192.168.112.53** for 10.9.8.0 to see what happens .. and use tcpdump on the vpn server .. ?
Tests Completed.

Test #1. Verify that the remote Windows client has correctly set up the pushed route to the 192.168.112.0/24 subnet.

Before VPN Connection (Windows client route print)

Code: Select all

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.123.2 192.168.123.111 10
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.56.0 255.255.255.0 On-link 192.168.56.1 276
192.168.56.1 255.255.255.255 On-link 192.168.56.1 276
192.168.56.255 255.255.255.255 On-link 192.168.56.1 276
192.168.123.0 255.255.255.0 On-link 192.168.123.111 266
192.168.123.111 255.255.255.255 On-link 192.168.123.111 266
192.168.123.255 255.255.255.255 On-link 192.168.123.111 266
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.56.1 276
224.0.0.0 240.0.0.0 On-link 192.168.123.111 266
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.56.1 276
255.255.255.255 255.255.255.255 On-link 192.168.123.111 266
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.123.2 Default
===========================================================================
After VPN Connection (Windows client route print)

Code: Select all

C:\>ipconfig
Ethernet adapter Local Area Connection 2:
Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::44cc:5041:5d8d:ae76%9
IPv4 Address. . . . . . . . . . . : 10.9.8.6
Subnet Mask . . . . . . . . . . . : 255.255.255.252
Default Gateway . . . . . . . . . :

[b]> added routes[/b]
IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.123.2 192.168.123.111 10
> 10.9.8.1 255.255.255.255 10.9.8.5 10.9.8.6 31
> 10.9.8.4 255.255.255.252 On-link 10.9.8.6 286
> 10.9.8.6 255.255.255.255 On-link 10.9.8.6 286
> 10.9.8.7 255.255.255.255 On-link 10.9.8.6 286
127.0.0.0 255.0.0.0 On-link 127.0.0.1 306
127.0.0.1 255.255.255.255 On-link 127.0.0.1 306
127.255.255.255 255.255.255.255 On-link 127.0.0.1 306
192.168.56.0 255.255.255.0 On-link 192.168.56.1 276
192.168.56.1 255.255.255.255 On-link 192.168.56.1 276
192.168.56.255 255.255.255.255 On-link 192.168.56.1 276
> 192.168.112.0 255.255.255.0 10.9.8.5 10.9.8.6 31
192.168.123.0 255.255.255.0 On-link 192.168.123.111 266
192.168.123.111 255.255.255.255 On-link 192.168.123.111 266
192.168.123.255 255.255.255.255 On-link 192.168.123.111 266
224.0.0.0 240.0.0.0 On-link 127.0.0.1 306
224.0.0.0 240.0.0.0 On-link 192.168.56.1 276
224.0.0.0 240.0.0.0 On-link 192.168.123.111 266
> 224.0.0.0 240.0.0.0 On-link 10.9.8.6 286
255.255.255.255 255.255.255.255 On-link 127.0.0.1 306
255.255.255.255 255.255.255.255 On-link 192.168.56.1 276
255.255.255.255 255.255.255.255 On-link 192.168.123.111 266
> 255.255.255.255 255.255.255.255 On-link 10.9.8.6 286
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.123.2 Default
===========================================================================
This appears to be correct. The Windows client has received the pushed route.

Test #2. Run Wireshark on the TUN interfaces and see whether packets are going into the tunnel.

Wireshark on TUN interface of remote Windows Client during ping to 192.168.112.53

Code: Select all

No. Time Source Destination Protocol Info
1 16:14:13.608219 10.9.8.6 192.168.112.53 ICMP Echo (ping) request (id=0x0001, seq(be/le)=21/5376, ttl=128)

Frame 1: 74 bytes on wire (592 bits), 74 bytes captured (592 bits)
Ethernet II, Src: 00:ff:dc:92:57:17 (00:ff:dc:92:57:17), Dst: 10.9.8.5 (00:ff:dd:92:57:17)
Internet Protocol, Src: 10.9.8.6 (10.9.8.6), Dst: 192.168.112.53 (192.168.112.53)
Internet Control Message Protocol

No. Time Source Destination Protocol Info
2 16:14:13.665459 10.9.8.1 10.9.8.6 ICMP Destination unreachable (Host administratively prohibited)

Frame 2: 102 bytes on wire (816 bits), 102 bytes captured (816 bits)
Ethernet II, Src: 10.9.8.5 (00:ff:dd:92:57:17), Dst: 00:ff:dc:92:57:17 (00:ff:dc:92:57:17)
Internet Protocol, Src: 10.9.8.1 (10.9.8.1), Dst: 10.9.8.6 (10.9.8.6)
Internet Control Message Protocol
< repeated 4 times >
Reading this output, the "host administratively prohibited" phrase caught my attention, as it implied to me that the system knew the proper routing, but that the packet was being administratively blocked (ie., by a firewall). So then I had to figure out which firewall, on which box within the route the packet was taking that was causing the problem. Looking back at my original post I had listed the firewall rules on the VPN box and speculated that I might need an accept rule on the Forward Chain. So I took a shot and changed the default setting on the Forward Chain of my VPN box to ACCEPT ALL, rather than DENY ALL. Once I made that change I could ping any box on the LAN behind the VPN box.

Now, I didn’t want to leave my firewall with a default Accept All setting on the forwarding chain, so I needed to identify a rule specific to the packet type / traffic that I want to allow. Some additional suggestions from folks on the mailing list pointed me in the right direction. The basic solution is to allow traffic using the TUN interface to be forwarded, which means accepting them on the input side of the interface and the output side of the interface (the definition of forwarding). The basic rules are listed below.

Code: Select all

iptables -A FORWARD -i tun+ -j ACCEPT
iptables -A FORWARD -o tun+ -j ACCEPT
These would need to be placed before any REJECT ALL rule on the forward chain. After adding the rules, the Forward chain on my VPN box now shows:

Code: Select all

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 1068  115K ACCEPT     all  --  tun0   any     anywhere             anywhere
 1068  117K ACCEPT     all  --  any    tun0    anywhere             anywhere
    0     0 REJECT     all  --  any    any     anywhere             anywhere            reject-with icmp-port-unreachable
After this firewall change, I am now able to access all boxes behind my VPN server; other servers and desktop systems. Examples below.

Pings from remote Windows Client connected to VPN.

Code: Select all

C:\HomeFiles>ping 192.168.112.53  (other server behind VPN server)
Pinging 192.168.112.53 with 32 bytes of data:
Reply from 192.168.112.53: bytes=32 time=57ms TTL=62
Reply from 192.168.112.53: bytes=32 time=53ms TTL=63
Reply from 192.168.112.53: bytes=32 time=54ms TTL=63
Reply from 192.168.112.53: bytes=32 time=56ms TTL=63
Ping statistics for 192.168.112.53:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 53ms, Maximum = 57ms, Average = 55ms

C:\HomeFiles>ping 192.168.112.101  (desktop box behind VPN server)
Pinging 192.168.112.101 with 32 bytes of data:
Reply from 192.168.112.101: bytes=32 time=57ms TTL=126
Reply from 192.168.112.101: bytes=32 time=59ms TTL=127
Reply from 192.168.112.101: bytes=32 time=55ms TTL=127
Reply from 192.168.112.101: bytes=32 time=57ms TTL=127
Ping statistics for 192.168.112.101:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 55ms, Maximum = 59ms, Average = 57ms
So in conclusion… After I had a working connection directly to my VPN server I have made two changes in my network that now allow full access to all systems behind the VPN server. Note that in my case, my network router is on a different box than my VPN server.

Change #1. Added a static route on the default gateway box that directs the VPN tunnel network traffic back to the VPN box; shown as the second one on the list below.

Code: Select all

[2.1.2-RELEASE][admin@pfgateway.mei.lan]/root(1): netstat -rn
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            xxx.yy.190.17      UGS         1 663197177   bge0
10.9.8.0/24        192.168.112.50     UGS         0        0    rl0
xxx.yy.190.16/28   link#1             U           0 49927480   bge0
xxx.yy.190.18      link#1             UHS         0        0    lo0
xxx.yy.190.19      link#1             UHS         0        0    lo0 =>
xxx.yy.190.19/32   link#1             U           0        0   bge0
xxx.yy.190.20      link#1             UHS         0        0    lo0 =>
xxx.yy.190.20/32   link#1             U           0        0   bge0
xxx.yy.190.21      link#1             UHS         0        0    lo0 =>
xxx.yy.190.21/32   link#1             U           0        0   bge0
xxx.yy.190.22      link#1             UHS         0        0    lo0 =>
xxx.yy.190.22/32   link#1             U           0        0   bge0
127.0.0.1          link#6             UH          0      560    lo0
192.168.112.0/24   link#2             U           0 848109484    rl0
192.168.112.11     link#2             UHS         0 99857580    lo0
Change #2. Added two firewall rules to accept the VPN traffic that is forwarded across the VPN box.

Code: Select all

iptables -A FORWARD -i tun+ -j ACCEPT
iptables -A FORWARD -o tun+ -j ACCEPT
That is all. I hope this detailed description helps anyone else that stumbles across it looking to solve a similar issue. Note that once I identified the root cause issue and the basic solution, I did not complete further diagnostic tests suggested by Gert, Morten, or debbie10t and don’t know what additional help those tests may have provided me, or provide to others in a similar situation. Thanks to all those that gave me guidance in my search for a solution.

Jeff

Locked