TAP Server running in Cloud - Cannot PING Server

solarblitz

Hello All.
I am creating a cloud resource to serve some IoT functions between several locations (thus the need for a TAP). My installation uses Docker to host a TUN server (which is working well - and selected docker containers are accessible from individual devices ~ phones) as well as a TAP server (on a different port w/ different TLS keys). I have a OpenWRT router inside my network (it is not the perimeter router) setup as a TAP client. I need a bridge because I want small, 'lightweight' IoT devices with wifi to send/receive MQTT packets to a MQTT broker in the cloud. The OpenWRT client connects successfully - but I cannot even ping between the OpenVPN client (on my inside router) and OpenVPN server (in a cloud based docker container). I have checked firewalls - and all seems to be at ACCEPT.. so I think I have a routing problem.

As OpenWRT uses a (nice) GUI to manipulate the configs - I'm a little confused on how to resolve my issue. I think I have zones setup (lan, wan, VPNEntry). I have forwarding rules (separate) in both directions for wan <==> lan <==> VPNEntry

I can ping the assigned VPN client address (192.168.9.128) from the OpenWRT and from my laptop (both when on OpenWRT LAN or on Perimeter Router LAN - so I think the static routes between my routers are working).

In the spirit of Halloween - I offer ' FrankenConfig' (after 2 weeks of study, head-banging, failed / reloaded configs - I'd really appreciate some expertise) !

my Server.conf (in the cloudbased Docker container)

dev tap0
server-bridge 192.168.9.1 255.255.255.0 192.168.9.128 192.168.9.254
route-gateway 192.168.9.1
ifconfig 192.168.9.1 255.255.255.0
verb 5
key /etc/openvpn/pki/private/vpn.server.key
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/vpn.server.crt
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/ta.key
key-direction 0
keepalive 10 119
persist-key
###  persist_tun  ###
client-config-dir ccd  ### this controls which locations have access to which resources
proto udp
# Rely on Docker to do port mapping, internally always 1195 for bridging
# https://openvpn.net/community-resources/ethernet-bridging/
port 1195
status /tmp/openvpn-status.log

user nobody
#group nogroup
comp-lzo

### Route Configurations Below
##route 192.168.176.0 255.255.255.0 192.168.9.1   #the  docker containers are 192.168.176.0/24

### Push Configurations Below
push "block-outside-dns"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"

the relevant docker-compose.yml file to start the TAP server. This is very similar to my TUN server which is working (port 1194).

  bridgevpn:
    container_name: openvpn-bridge
    privileged: true
    image: kylemanna/openvpn
    cap_add:
      - NET_ADMIN
    restart: always
    ports:
       - "1195:1195/udp"
    volumes:
       - /mnt/volume/openvpn/bridge_openvpn_data:/etc/openvpn
       - /mnt/volume/datawall/logs/bridge_vpn:/tmp
    networks:
      default:
        ipv4_address: 192.168.176.10

ifconfig of the TAP bridge server (when running). I honestly dont know how to bridge from the 192.168.9.0/25 into the 192.168.176/24
the 192.168.9.1 is the same as 192.168.176.10. Is there a brIDGE interface missing ? I didn't add anything on the TUN server - docker took care of it - all of my target containers see TUN users as the gateway (in this case 192.168.176.10) container.

eth0      Link encap:Ethernet  HWaddr 02:42:C0:A8:B0:0A  
          inet addr:192.168.176.10  Bcast:192.168.176.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:394 errors:0 dropped:0 overruns:0 frame:0
          TX packets:634 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:62687 (61.2 KiB)  TX bytes:74261 (72.5 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

tap0      Link encap:Ethernet  HWaddr 36:2E:3E:9F:FE:D2  
          inet addr:192.168.9.1  Bcast:192.168.9.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:101 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:10179 (9.9 KiB)  TX bytes:504 (504.0 B)

On my OperWRT Router I have the following ifconfig:

br-Bristol Link encap:Ethernet  HWaddr BA:E5:D2:7A:1F:06  
          inet6 addr: fe80::b8e5:d2ff:fe7a:1f06/64 Scope:Link      #  Bristol is the SSID for the USB  Wifi AccessPoint
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:746 (746.0 B)

br-VPNEntry Link encap:Ethernet  HWaddr B6:8E:53:AD:C9:C8      #  this is an interface that I manually created - intended to be my path to the VPN
          inet addr:192.168.35.2  Bcast:192.168.35.255  Mask:255.255.255.0   #   192.168.35.2 <==> 192.168.9.128
          inet6 addr: fe80::b48e:53ff:fead:c9c8/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:746 (746.0 B)

br-WAN    Link encap:Ethernet  HWaddr BA:E5:82:B9:12:5D  
          inet addr:192.168.87.10  Bcast:192.168.87.255  Mask:255.255.255.0     # the perimeter router is 192.168.87.1  this is the reserved address on upstream
          inet6 addr: fe80::b8e5:82ff:feb9:125d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:64376 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29715 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:34443452 (32.8 MiB)  TX bytes:6598121 (6.2 MiB)

br-lan    Link encap:Ethernet  HWaddr 00:13:25:B1:97:9D  
          inet addr:192.168.35.1  Bcast:192.168.35.255  Mask:255.255.255.0   #this is the OpenWRT LAN  subnet  it is successfully bridged to Bristol
          inet6 addr: fd00:ab:cd::1/60 Scope:Global
          inet6 addr: fe80::213:25ff:feb1:979d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:53452 errors:0 dropped:0 overruns:0 frame:0
          TX packets:78293 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:8365229 (7.9 MiB)  TX bytes:68225401 (65.0 MiB)

docker0   Link encap:Ethernet  HWaddr 02:42:AC:68:8D:D9  
          inet addr:172.17.0.1  Bcast:172.17.255.255  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

dummy0    Link encap:Ethernet  HWaddr 8A:85:09:C9:AD:74  
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

erspan0   Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1450  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr BA:E5:82:B9:12:5D  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:73989 errors:0 dropped:0 overruns:0 frame:0
          TX packets:30309 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:39517771 (37.6 MiB)  TX bytes:6637085 (6.3 MiB)
          Interrupt:37 

eth1      Link encap:Ethernet  HWaddr BA:E5:D2:7A:1F:06  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:1492 (1.4 KiB)

gre0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          NOARP  MTU:1476  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

gretap0   Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1462  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ifb0      Link encap:Ethernet  HWaddr BE:0B:8B:C9:9A:86  
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

ifb1      Link encap:Ethernet  HWaddr AE:2F:8E:CF:24:FD  
          BROADCAST NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:37999 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37999 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3309841 (3.1 MiB)  TX bytes:3309841 (3.1 MiB)

on the OpenWRT, /etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd00:ab:cd::/48'

config interface 'lan'
	option ifname 'eth1'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ipaddr '192.168.35.1'
	option delegate '0'
	option type 'bridge'

config interface 'WAN'
	option proto 'dhcp'
	option hostname 'HouseWRT'
	option type 'bridge'

config interface 'loopback'
	option hostname 'HouseWRT'
	option type 'bridge'
	option type 'bridge'
	option ifname 'eth0'

config interface 'VPNEntry'
	option proto 'static'
	option netmask '255.255.255.0'
	option ipaddr '192.168.35.2'
	option type 'bridge'
	option ifname 'eth1 tap0'

config interface 'Bristol'
	option proto 'none'
	option type 'bridge'
	option ifname 'eth1'

config route
	option target '192.168.9.0/24'
	option gateway '192.168.35.2'
	option netmask '255.255.255.0'
	option interface 'VPNEntry'

on the OpenWRT, /etc/config/firewall

onfig defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option network 'lan'

config zone
	option name 'wan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option mtu_fix '1'
	option network 'WAN'
	option masq '1'

config forwarding
	option dest 'wan'
	option src 'lan'

config forwarding
	option dest 'lan'
	option src 'wan'

config zone
	option name 'VPN'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option network 'VPNEntry'
	option output 'ACCEPT'

config forwarding
	option dest 'lan'
	option src 'VPN'

config zone
	option name 'OuttoVPN'
	option network 'lan'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option masq '1'
	option output 'ACCEPT'

config forwarding
	option dest 'VPN'
	option src 'OuttoVPN'

the client side .ovpn file (without crypto)

client
nobind
dev tap0
remote-cert-tls server
persist-key
route 192.168.176.0 255.255.255.0 192.168.9.4
route 192.168.9.0 255.255.255.0 192.168.35.2
remote <public IP address of cloud server>  1195 udp
comp-lzo


persist-tun
user nobody
<tls-auth></tls-auth>
redirect-gateway def1

TinCanTech

It would appear that you have successfully installed openvpn where ever you need it.

solarblitz

Thank you, @TinCanTech. I have seen your previous support on this forum and was hoping I may benefit from your expertise. Do you have any thoughts about my routing issue ? It seems odd that I can't ping from client to server.

solarblitz

Does anyone have thoughts on why I can't ping through this tunnel ? I'm weeks into this and not getting any closer. It seems that the connection is established. I don't think I have any firewall rules preventing. The Br-VPNEntry uses the tap0 interface .. but I can't ping from client to server or from the server back to the client. Any help would be greatly appreciated.

TinCanTech

OpenVPN is working normally, the problem is your network.

If all else fails then you can contact me privately (Fees will apply):
tincantech at protonmail dot com

TinCanTech

This is left as an exercise for subsequent readers:

What is wrong with the bridge configurations above?

Hint: This is documented here on openvpn.net

Info: The setup above does not require any bridges.