Odd routing problem

Last Chance

I have two machines connecting via Open VPN. Works great. EXCEPT, the Server can not ping the Clients physical lan. Client can ping the server's physical lan address. I need this to work as I have an Active Directory running on this an all IP's must resolve via DNS. So I can ping from the client, the server's physical address 192.168.3.4, but from the server, I can only ping the Client's TAP address 10.8.0.6, not 192.168.1.12.. Routing tables look good to me.

Server Lan 192.168.3.0 (I know not ideal , will work for my setup)
Client Lan 192.168.1.1 (I know, bad class "C". Will work for what I am doing).

Client Config (Minus host name)
client
dev tun
proto udp
remote *private* 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca "c:\\openvpn\\config\\ca.crt"
cert "c:\\openvpn\\config\\Private.crt"
key "c:\\openvpn\\config\\Private.key"
ns-cert-type server
comp-lzo
verb 3

Host name and cert names changed\ for privacy, but the network connects and pings each other, and all Windows services work over the Open VPN adapters, so not an issue.

Server config:

port 1194
proto udp
dev tun
ca "c:\\OpenVpn\\config\\ca.crt"
cert "c:\\OpenVpn\\config\\vicserver.crt"
key "c:\\OpenVpn\\config\\vicserver.key" # This file should be kept secret
dh "c:\\OpenVpn\\config\\dh1024.pem"
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "route 192.168.3.0 255.255.255.0"
route 192.168.1.0 255.255.255.0
push "route 192.168.1.0 255.255.255.0"
client-config-dir "c:\\openvpn\\config\\ccd"
push "dhcp-option DNS 192.168.3.4"
client-to-client
keepalive 5 20
comp-lzo
persist-key
persist-tun
status openvpn-status.log
verb 3
route method exe
route-delay

And the client config file

iroute 192.168.1.0 255.255.255.0

Routing table from server:

C:\Users\Administrator>route print
===========================================================================
Interface List
14...00 15 5d 03 54 03 ......Microsoft Hyper-V Network Adapter
4...00 ff 24 94 1b db ......TAP-Windows Adapter V9
1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.3.254 192.168.3.4 281
10.8.0.0 255.255.255.0 10.8.0.2 10.8.0.1 35
10.8.0.0 255.255.255.252 On-link 10.8.0.1 291
10.8.0.1 255.255.255.255 On-link 10.8.0.1 291
10.8.0.3 255.255.255.255 On-link 10.8.0.1 291
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
192.168.1.0 255.255.255.0 10.8.0.2 10.8.0.1 35
192.168.3.0 255.255.255.0 On-link 192.168.3.4 281
192.168.3.4 255.255.255.255 On-link 192.168.3.4 281
192.168.3.255 255.255.255.255 On-link 192.168.3.4 281
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 10.8.0.1 291
224.0.0.0 240.0.0.0 On-link 192.168.3.4 281
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 10.8.0.1 291
255.255.255.255 255.255.255.255 On-link 192.168.3.4 281
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.3.254 Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 331 ::1/128 On-link
1 331 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

Routing table from client:

===========================================================================
Interface List
13...00 ff f8 6e 5e 01 ......TAP-Windows Adapter V9
12...00 15 5d 01 0a 01 ......Microsoft Hyper-V Network Adapter
1...........................Software Loopback Interface 1
===========================================================================

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination Netmask Gateway Interface Metric
0.0.0.0 0.0.0.0 192.168.1.1 192.168.1.12 301
10.8.0.0 255.255.255.0 10.8.0.5 10.8.0.6 35
10.8.0.4 255.255.255.252 On-link 10.8.0.6 291
10.8.0.6 255.255.255.255 On-link 10.8.0.6 291
10.8.0.7 255.255.255.255 On-link 10.8.0.6 291
127.0.0.0 255.0.0.0 On-link 127.0.0.1 331
127.0.0.1 255.255.255.255 On-link 127.0.0.1 331
127.255.255.255 255.255.255.255 On-link 127.0.0.1 331
192.168.1.0 255.255.255.0 On-link 192.168.1.12 301
192.168.1.12 255.255.255.255 On-link 192.168.1.12 301
192.168.1.255 255.255.255.255 On-link 192.168.1.12 301
192.168.3.0 255.255.255.0 10.8.0.5 10.8.0.6 35
224.0.0.0 240.0.0.0 On-link 127.0.0.1 331
224.0.0.0 240.0.0.0 On-link 10.8.0.6 291
224.0.0.0 240.0.0.0 On-link 192.168.1.12 301
255.255.255.255 255.255.255.255 On-link 127.0.0.1 331
255.255.255.255 255.255.255.255 On-link 10.8.0.6 291
255.255.255.255 255.255.255.255 On-link 192.168.1.12 301
===========================================================================
Persistent Routes:
Network Address Netmask Gateway Address Metric
0.0.0.0 0.0.0.0 192.168.1.1 Default
===========================================================================

IPv6 Route Table
===========================================================================
Active Routes:
If Metric Network Destination Gateway
1 331 ::1/128 On-link
1 331 ff00::/8 On-link
===========================================================================
Persistent Routes:
None

Pippin

Enabled routing mode on the client?

SomeGuy

Yes, as @Pippin suggest, ensure that the client has support for IP-forwarding/routing between interfaces.

Summary:
* Server using 192.168.3.4/24 on a LAN interface with Default GW: 192.168.3.254
* Client using 192.168.1.12/24 on a LAN interface with Default GW: 192.168.1.1
* Through some cloud/network/Routers/NAT, 192.168.1.1 (<----->) 192.168.3.254
* Client established OpenVPN link through to server over this network and link comes up.
* TUN network is 10.8.0.0/24
* Server tun interface IP: 10.8.0.1
* Client tun: 10.8.0.4/30, using IP 10.8.0.6 for "self" and 10.8.0.5 for server-end.
* You have a CCD for the client to include iroute when connected by client

The CCD file, on the server with these contents:

iroute 192.168.1.0 255.255.255.0

... is it stored in

client-config-dir "c:\\openvpn\\config\\ccd"

?

Does the filename it is saved under match the name of the client's key that is connecting, and if server is *nix, matching case?

Have you checked your logs to see if the the client CCD you want to have loaded on that client's connection actually is being loaded when the client connects?

Maybe something like:

SERVER_NAME/SERVER_IP:SERVER_PORT OPTIONS IMPORT: reading client specific options from: c:\\openvpn\\config\\ccd\\CLIENT_NAME

If that is missing, maybe the filename you saved it as does not match the name of the incoming client. Check the logs for the name of the client, and move the filename to match. I know most Windows FS allow for case -insensitive storage of file name, and a request to open a file "FileName.txt" will open a file "filename.txt" but if you see no logging that shows the CCD file was actually opened, and there is a case difference, try changing the name to a temp file, then rename it again to match the case. (Most apps and calls to open files on windows should not care, but it is worth considering. Make the CDD filename match the case of of the name of the client that connect, exactly as the name appears in the logs: only worth a try if there is no logged indication that the CCD was loaded.)

Does the client have any firewall rules that might block such traffic even if it is configured by OpenVPN to be forwarded/routed?

Last Chance

Ok, facepalm, appeared to be an issue of Case Sensitivity for the CCD file. Did not expect this for Windows.

Next challenge, and I'll donate 20 bucks to Wikipedia if someone can answer me this.

Pinging through the tunnel (From 193.168.3.4 to 10.8.0.6), I get good steady pings 20-25MS, which is on par with what I get pinging router to router outside of the tunnel.

HOWEVER, pinging through the tunnel from 192.168.3.4 I get erratic pings, 233MS-22MS, all over the place up to 400+.. Anythoughts? I don't think it's an issue of MTU, both the host and client are modern very capable processors so I do not think compression is the issue.

Here is the server log on startup:

Wed Dec 21 09:29:42 2016 OpenVPN 2.3.14 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Dec 7 2016
Wed Dec 21 09:29:42 2016 Windows version 6.2 (Windows 8 or greater) 64bit
Wed Dec 21 09:29:42 2016 library versions: OpenSSL 1.0.2i 22 Sep 2016, LZO 2.09
Wed Dec 21 09:29:42 2016 Diffie-Hellman initialized with 1024 bit key
Wed Dec 21 09:29:42 2016 Socket Buffers: R=[65536->65536] S=[65536->65536]
Wed Dec 21 09:29:42 2016 ROUTE_GATEWAY 192.168.3.254/255.255.255.0 I=14 HWADDR=00:15:5d:03:54:03
Wed Dec 21 09:29:44 2016 RESOLVE: Cannot resolve host address: method: No such host is known.
Wed Dec 21 09:29:44 2016 OpenVPN ROUTE: failed to parse/resolve route for host/network: method
Wed Dec 21 09:29:44 2016 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Wed Dec 21 09:29:44 2016 open_tun, tt->ipv6=0
Wed Dec 21 09:29:44 2016 TAP-WIN32 device [Ethernet 2] opened: \\.\Global\{24941BDB-D0D1-41FC-9CED-F7825936CE4B}.tap
Wed Dec 21 09:29:44 2016 TAP-Windows Driver Version 9.21
Wed Dec 21 09:29:44 2016 Notified TAP-Windows driver to set a DHCP IP/netmask of 10.8.0.1/255.255.255.252 on interface {24941BDB-D0D1-41FC-9CED-F7825936CE4B} [DHCP-serv: 10.8.0.2, lease-time: 31536000]
Wed Dec 21 09:29:44 2016 Sleeping for 10 seconds...
Wed Dec 21 09:29:54 2016 Successful ARP Flush on interface [4] {24941BDB-D0D1-41FC-9CED-F7825936CE4B}
Wed Dec 21 09:29:54 2016 C:\Windows\system32\route.exe ADD 192.168.1.0 MASK 255.255.255.0 10.8.0.2
Wed Dec 21 09:29:54 2016 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=35 and dwForwardType=4
Wed Dec 21 09:29:54 2016 Route addition via IPAPI succeeded [adaptive]
Wed Dec 21 09:29:54 2016 C:\Windows\system32\route.exe ADD 10.8.0.0 MASK 255.255.255.0 10.8.0.2
Wed Dec 21 09:29:54 2016 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=35 and dwForwardType=4
Wed Dec 21 09:29:54 2016 Route addition via IPAPI succeeded [adaptive]
Wed Dec 21 09:29:54 2016 UDPv4 link local (bound): [undef]
Wed Dec 21 09:29:54 2016 UDPv4 link remote: [undef]
Wed Dec 21 09:29:54 2016 MULTI: multi_init called, r=256 v=256
Wed Dec 21 09:29:54 2016 IFCONFIG POOL: base=10.8.0.4 size=62, ipv6=0
Wed Dec 21 09:29:54 2016 ifconfig_pool_read(), in='blenkinsop,10.8.0.4', TODO: IPv6
Wed Dec 21 09:29:54 2016 succeeded -> ifconfig_pool_set()
Wed Dec 21 09:29:54 2016 IFCONFIG POOL LIST
Wed Dec 21 09:29:54 2016 blenkinsop,10.8.0.4
Wed Dec 21 09:29:54 2016 Initialization Sequence Completed

Here is the client log.

Wed Dec 21 09:30:51 2016 OpenVPN 2.3.14 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [PKCS11] [IPv6] built on Dec 7 2016
Wed Dec 21 09:30:51 2016 Windows version 6.2 (Windows 8 or greater) 64bit
Wed Dec 21 09:30:51 2016 library versions: OpenSSL 1.0.2i 22 Sep 2016, LZO 2.09
Wed Dec 21 09:30:51 2016 Socket Buffers: R=[65536->65536] S=[65536->65536]
Wed Dec 21 09:30:51 2016 UDPv4 link local: [undef]
Wed Dec 21 09:30:51 2016 UDPv4 link remote: [AF_INET]184.69.103.110:1194
Wed Dec 21 09:30:51 2016 TLS: Initial packet from [AF_INET]184.69.103.110:1194, sid=dd1dacc1 1b7e2a21
Wed Dec 21 09:30:51 2016 VERIFY OK: depth=1, C=CA, ST=BC, L=Woodsmere, O=Woodsmere, emailAddress=me@woodsmere.ca
Wed Dec 21 09:30:51 2016 VERIFY OK: nsCertType=SERVER
Wed Dec 21 09:30:51 2016 VERIFY OK: depth=0, C=CA, ST=BC, O=Woodsmere, CN=Vicserver, emailAddress=me@woodsmere.ca
Wed Dec 21 09:30:52 2016 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Wed Dec 21 09:30:52 2016 WARNING: INSECURE cipher with block size less than 128 bit (64 bit). This allows attacks like SWEET32. Mitigate by using a --cipher with a larger block size (e.g. AES-256-CBC).
Wed Dec 21 09:30:52 2016 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Wed Dec 21 09:30:52 2016 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Wed Dec 21 09:30:52 2016 WARNING: INSECURE cipher with block size less than 128 bit (64 bit). This allows attacks like SWEET32. Mitigate by using a --cipher with a larger block size (e.g. AES-256-CBC).
Wed Dec 21 09:30:52 2016 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Wed Dec 21 09:30:52 2016 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 1024 bit RSA
Wed Dec 21 09:30:52 2016 [Vicserver] Peer Connection Initiated with [AF_INET]184.69.103.110:1194
Wed Dec 21 09:30:54 2016 SENT CONTROL [Vicserver]: 'PUSH_REQUEST' (status=1)
Wed Dec 21 09:30:54 2016 PUSH: Received control message: 'PUSH_REPLY,route 192.168.3.0 255.255.255.0,dhcp-option DNS 192.168.3.4,route 10.8.0.0 255.255.255.0,topology net30,ping 5,ping-restart 20,ifconfig 10.8.0.6 10.8.0.5'
Wed Dec 21 09:30:54 2016 OPTIONS IMPORT: timers and/or timeouts modified
Wed Dec 21 09:30:54 2016 OPTIONS IMPORT: --ifconfig/up options modified
Wed Dec 21 09:30:54 2016 OPTIONS IMPORT: route options modified
Wed Dec 21 09:30:54 2016 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Wed Dec 21 09:30:54 2016 ROUTE_GATEWAY 192.168.1.1/255.255.255.0 I=12 HWADDR=00:15:5d:01:0a:01
Wed Dec 21 09:30:54 2016 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Wed Dec 21 09:30:54 2016 open_tun, tt->ipv6=0
Wed Dec 21 09:30:54 2016 TAP-WIN32 device [Ethernet 2] opened: \\.\Global\{F86E5E01-2A73-4E1E-A199-BD35F64611BF}.tap
Wed Dec 21 09:30:54 2016 TAP-Windows Driver Version 9.21
Wed Dec 21 09:30:54 2016 Notified TAP-Windows driver to set a DHCP IP/netmask of 10.8.0.6/255.255.255.252 on interface {F86E5E01-2A73-4E1E-A199-BD35F64611BF} [DHCP-serv: 10.8.0.5, lease-time: 31536000]
Wed Dec 21 09:30:54 2016 Successful ARP Flush on interface [13] {F86E5E01-2A73-4E1E-A199-BD35F64611BF}
Wed Dec 21 09:30:59 2016 TEST ROUTES: 2/2 succeeded len=2 ret=1 a=0 u/d=up
Wed Dec 21 09:30:59 2016 C:\Windows\system32\route.exe ADD 192.168.3.0 MASK 255.255.255.0 10.8.0.5
Wed Dec 21 09:30:59 2016 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=35 and dwForwardType=4
Wed Dec 21 09:30:59 2016 Route addition via IPAPI succeeded [adaptive]
Wed Dec 21 09:30:59 2016 C:\Windows\system32\route.exe ADD 10.8.0.0 MASK 255.255.255.0 10.8.0.5
Wed Dec 21 09:30:59 2016 ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=35 and dwForwardType=4
Wed Dec 21 09:30:59 2016 Route addition via IPAPI succeeded [adaptive]
Wed Dec 21 09:30:59 2016 Initialization Sequence Completed

Pippin

Wed Dec 21 09:29:44 2016 RESOLVE: Cannot resolve host address: method: No such host is known. 
Wed Dec 21 09:29:44 2016 OpenVPN ROUTE: failed to parse/resolve route for host/network: method

Looks like DNS problem.....?
Ping to DNS is OK when server is down?

SomeGuy

@pippin : good catch on the error reported WRT DNS, but based on context and error message I think it is a complaint about this line in the config:

route method exe

I think the original posted config for server is missing a "-":

route-method exe

Once that line is changed, and OpenVPN server restarted, that error message should go away along with the other error message.

@pippin : Good eye for observing the error you highlighted. :-)
(My praise is genuine: I would have dismissed that as unrelated to this random latency issue; you found it and another error will likely be fixed in this user's server config.)

As for the erratic timing on ping when passing through the host, that could be caused by many things. The first guess I would have would be a "traffic shaping tool." On the box that is the gateway through which the erratic latency is being observed, do you have any traffic shaping network tools enabled? Any "live scan" tools that check for "attacks" such as found in some AV tools that check network traffic? One example for shaping in windows that could be on the network interface feature in windows called "QoS" ( Check out https://technet.microsoft.com/en-us/library/dd759093%28v=ws.11%29.aspx )

One thing you could try is to disable "QoS" on the windows virtual network device used by the vpn, *and* on the LAN that is attached to the VPN host through which you are trying to ping a network. Many schedulers consider icmp ping, NTP, and other similar traffic lower priority. Some will just drop the first NTP service packets entirely, but allow later packets to pass. DHCP relay is another one, where first packets may be dropped by schedulers, but the next one(s) allowed -- dhcp relay is uncommon through routers in small networks, and very uncommon through vpn tunnels -- these are just examples of protocols often hit by traffic shaping issues.)

Check and disable these QoS features on interfaces participating in the path from one end to the other, and disable any active network checks by malware scanners, or IDS (Intrusion Detection Systems) , or IPS (Intrusion Prevention Systems) and similar, and then repeat the test. if that removes the latency issue, then you found the cause and should consider re-enabling these, as they are often useful/helpful for most use cases.

SomeGuy

(I tried to edit my post instead of posting new, but it looks like I exceeded my timeout for being able to edit posts... 15 minutes?)
(So, a new post:)

You may also want to consider commenting out the fixed "route-method exe" from the server config: if there was an error parsing it, then it was not loaded by the server, but the client could still connect without it, and commenting-it-out probably makes your OpenVPN server config more compatible for more, and different clients.