Can't connect to devices on server-side VLAN

[toothless92]

I am a networking noob and could use some help.

I am trying to set up a TAP connection using OpenVPN to access devices that are on a VLAN on my remote server. Right now, I can connect and ping the interface (172.20.0.101), but I can't ping anything on the VLAN (like 172.20.0.2). I don't care about the VLAN coming accross my VPN connection, I just want to be able to see those devices.

The "public" IP for the server is 10.249.2.93 (this is actually a LAN I connect to over another VPN).

The subnet I want to be able to talk to is 172.20.0.0/24, which is on vlan.3072, according to ifconifg. The associated physical interface is enp3s0.

I had this working on a different server that does not use a VLAN, i.e. I was able to successfully connect to devices on a regular LAN behind that server, so I think the VLAN is the issue.

Can anyone point me in the right direction? Any help is very much appreciated, as I am going losing my mind =P

Server is running Ubuntu 18.04 LTS, OpenVPN 2.4.4

Client is running Windows, OpenVPN GUI 11.14.0.0

server.conf:

Code: Select all

port 1194
    proto udp
    dev tap0
    ca ca.crt
    cert server.crt
    key server.key  # This file should be kept secret
    ifconfig-pool-persist /var/log/openvpn/ipp.txt
    server-bridge 172.20.0.101 255.255.255.0 172.20.0.200 172.20.0.240
    keepalive 10 120
    tls-auth ta.key 0 # This file is secret
    key-direction 0
    cipher AES-256-CBC
    auth SHA256
    persist-key
    persist-tun
    status /var/log/openvpn/openvpn-status.log
    verb 4
    explicit-exit-notify 1
    tls-auth ta.key 0
    key-direction 0
    cipher AES-256-CBC
    auth SHA256
    dh dh.pem
    user nobody
    group nogroup
    cert server.crt
    key server.key

clinet.ovpn:

Code: Select all

client
    dev tap
    proto udp
    remote 10.249.2.93 1194
    resolv-retry infinite
    nobind
    user nobody
    group nogroup
    persist-key
    persist-tun
    remote-cert-tls server
    cipher AES-256-CBC
    auth SHA256
    key-direction 1
    script-security 2
    verb 4
    # cert and keys omitted

ufw rules:

Code: Select all

sudo nano /etc/ufw/before.rules
    
    #change:
    *nat
    :POSTROUTING ACCEPT [0:0]
    -A POSTROUTING -s 172.20.0.0/24 -o br0 -j MASQUERADE
    COMMIT

forwarding:

Code: Select all

sudo nano /etc/sysctl.conf
    
    #change:    
    net.ipv4.ip_forward=1
    
    sudo sysctl -p

rules updates:

Code: Select all

 sudo ufw allow 1194/udp
    sudo ufw allow OpenSSH
    sudo ufw disable
    sudo ufw enable
    sudo iptables -A INPUT -i tap0 -j ACCEPT
    sudo iptables -A INPUT -i br0 -j ACCEPT
    sudo iptables -A FORWARD -i br0 -j ACCEPT

bridge-start script:

Code: Select all

# Define Bridge Interface
    br="br0"
    
    # Define list of TAP interfaces to be bridged,
    # for example tap="tap0 tap1 tap2".
    tap="tap0"
    
    # Define physical ethernet interface to be bridged
    # with TAP interface(s) above.
    eth="enp3s0"
    eth_ip="fe80::a5a:11ff:fe33:e6e3"
    eth_netmask="255.255.255.0"
    eth_broadcast="172.20.0.255"
    
    for t in $tap; do
        openvpn --mktun --dev $t
    done
    
    brctl addbr $br
    brctl addif $br $eth
    
    for t in $tap; do
        brctl addif $br $t
    done
    
    for t in $tap; do
        ifconfig $t 0.0.0.0 promisc up
    done
    
    ifconfig $eth 0.0.0.0 promisc up
    
    ifconfig $br inet6 add fe80::a5a:11ff:fe33:e6e3/64 up
    ifconfig br0:0 172.20.0.101 netmask 255.255.255.0 broadcast 172.20.0.255 up

ifconfig (not showing the br0 or tap0 because I can't start them right this minute, and also there are a ton of other veth* I omitted):

Code: Select all

docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
            inet6 fe80::42:60ff:fe8c:45de  prefixlen 64  scopeid 0x20<link>
            ether 02:42:60:8c:45:de  txqueuelen 0  (Ethernet)
            RX packets 74394  bytes 900967774 (900.9 MB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 63778  bytes 901066960 (901.0 MB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 10.249.2.93  netmask 255.255.240.0  broadcast 10.249.15.255
            inet6 fe80::c600:adff:fe94:e303  prefixlen 64  scopeid 0x20<link>
            ether c4:00:ad:94:e3:03  txqueuelen 1000  (Ethernet)
            RX packets 960832  bytes 1168467699 (1.1 GB)
            RX errors 0  dropped 166  overruns 0  frame 0
            TX packets 535763  bytes 73093159 (73.0 MB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
            device interrupt 16  memory 0xb2500000-b2520000
    
    enp1s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
            ether 08:5a:11:33:e8:00  txqueuelen 1000  (Ethernet)
            RX packets 0  bytes 0 (0.0 B)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 0  bytes 0 (0.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    enp3s0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
            inet6 fe80::a5a:11ff:fe33:e6e3  prefixlen 64  scopeid 0x20<link>
            ether 08:5a:11:33:e6:e3  txqueuelen 1000  (Ethernet)
            RX packets 9900475  bytes 7053325760 (7.0 GB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 10464489  bytes 8238594873 (8.2 GB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    kube-bridge: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1480
            inet 10.244.0.1  netmask 255.255.255.0  broadcast 10.244.0.255
            inet6 fe80::841:c2ff:fed1:3642  prefixlen 64  scopeid 0x20<link>
            ether 0a:41:c2:d1:36:42  txqueuelen 1000  (Ethernet)
            RX packets 2110720  bytes 234803960 (234.8 MB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 2264621  bytes 419964189 (419.9 MB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1000  (Local Loopback)
            RX packets 8383864  bytes 4038115926 (4.0 GB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 8383864  bytes 4038115926 (4.0 GB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    veth83317517: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1480
            inet6 fe80::d4c3:65ff:fe76:7592  prefixlen 64  scopeid 0x20<link>
            ether d6:c3:65:76:75:92  txqueuelen 0  (Ethernet)
            RX packets 28251  bytes 7794531 (7.7 MB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 29314  bytes 3608931 (3.6 MB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    vlan.2560: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 172.19.0.1  netmask 255.255.128.0  broadcast 172.19.127.255
            inet6 fe80::a5a:11ff:fe33:e6e3  prefixlen 64  scopeid 0x20<link>
            ether 08:5a:11:33:e6:e3  txqueuelen 1000  (Ethernet)
            RX packets 70170  bytes 25019840 (25.0 MB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 179  bytes 17252 (17.2 KB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    vlan.2816: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 172.19.130.100  netmask 255.255.255.0  broadcast      172.19.130.255
            inet6 fe80::a5a:11ff:fe33:e6e3  prefixlen 64  scopeid 0x20<link>
            ether 08:5a:11:33:e6:e3  txqueuelen 1000  (Ethernet)
            RX packets 1243206  bytes 65936318 (65.9 MB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 1194999  bytes 99652577 (99.6 MB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    
    vlan.3072: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
            inet 172.20.0.100  netmask 255.255.255.0  broadcast 172.20.0.255
            inet6 fe80::a5a:11ff:fe33:e6e3  prefixlen 64  scopeid 0x20<link>
            ether 08:5a:11:33:e6:e3  txqueuelen 1000  (Ethernet)
            RX packets 8544935  bytes 6820405811 (6.8 GB)
            RX errors 0  dropped 511  overruns 0  frame 0
            
            TX packets 9265376  bytes 8138226312 (8.1 GB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

except from ip a:

Code: Select all

 7: vlan.3072@enp3s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
        link/ether 08:5a:11:33:e6:e3 brd ff:ff:ff:ff:ff:ff
        inet 172.20.0.100/24 brd 172.20.0.255 scope global vlan.3072
           valid_lft forever preferred_lft forever
        inet6 fe80::a5a:11ff:fe33:e6e3/64 scope link
           valid_lft forever preferred_lft forever

[TinCanTech]

Not only bridged but also:

Code: Select all

    docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500

And:

Code: Select all

    eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    enp1s0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
    enp3s0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
    kube-bridge: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1480
    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
    veth83317517: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1480
    vlan.2560: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    vlan.2816: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    vlan.3072: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500

Have fun!

If you require professional help then I am available for hire.

[toothless92]

Hi TinCanTech, thanks for the reply!

I'm sorry, I couldn't quite understand your post (I am still learning about network setups, I am really a mechanical engineer). Could you elaborate on what you meant by "not only bridged but also..."?

[TinCanTech]

Could you elaborate on what you meant by "not only bridged but also..."?

You have a very complicated network stack. That has nothing to-do with Openvpn.

[toothless92]

I see, bummer.

Could I just trouble you for a couple quick questions? I am trying to educate myself about some of this, but I am coming up a bit short:

- Should I (theoretically) be able to see the VLAN traffic by bridging my tap0 with the NIC enp3s0? Is there anything special I need to do since the packets are tagged for a VLAN?
- Would doing the above bridge mess up the processes on the server that are already using the VLAN? Or are the packets "duplicated", like on a physical unmanaged switch?

[toothless92]

@Stephanie_Sy - you might be it the wrong thread

@TinCanTech - here is some updated info
I did some tcpdump monitoring on tap0, br0, enp3s0, and vlan.3072, here is what I found:

when I ping 172.20.0.100 (the vlan interface) from 172.20.0.200 (the client):
tap0 - 200-->100 ping request
br0 - 200-->100 ping request
ep3s0 - who-has 200 tell 100
vlan.2072 - who-has 200 tell 100

when I ping 172.20.0.1 (a device behind the server) from 172.20.0.200 (the client):
tap0 - who-has 1 tell 200
br0 - who-has 1 tell 200
enp3s0 - who-has 1 tell 200
vlan.3072 - who-has 1 tell 200

So:
- the ping request for 100 gets lost between br0 and ep3s0.
- the ping request for 1 never gets anywhere

If there is anything to be gleaned from that, any advice is appreciated, thanks!