bridgint networks with both TCP and UDP

[trex_daemon]

Hello,
I have an existing Openvpn setup with 4 network. I'm running them on routers (server : ddwrt, clients: tomato+ddwrt) and I am bridging the networks together.
The idea is that each network should access the internet via it's own gateway, only the network traffic between the clietns goes through the VPN.
Suddenly one of them fails to connect to the server any more
I've traced down and it looks like it's not working any more over UDP. I've tried with TCP, it connects successfully, but I can't ping anything.
What I want to achieve is to connect the 3rd computer via TCP and the others via UDP (as UDP is faster)

Here's the setup:

Server:

Code: Select all

openvpn --mktun --dev tap0
brctl addif br0 tap0
ifconfig tap0 0.0.0.0 promisc up
sleep 5
ln -s /usr/sbin/openvpn /tmp/myvpn
/tmp/myvpn --config openvpn.conf

/tmp/myvpn --config openvpntcp.conf

route add -net 192.168.1.0/24 dev br0

server config1 (UDP)
openvpn.conf

Code: Select all

# Tunnel options
mode server       # Set OpenVPN major mode
proto udp         # Setup the protocol (server)
port 1194         # TCP/UDP port number
dev tap0          # TUN/TAP virtual network device
keepalive 15 60   # Simplify the expression of --ping 
daemon            # Become a daemon after all initialization
verb 3            # Set output verbosity to n 
#comp-lzo          # Use fast LZO compression

# OpenVPN server mode options
client-to-client  # tells OpenVPN to internally route client-to-client traffic 
duplicate-cn      # Allow multiple clients with the same common name

# TLS Mode Options
tls-server        # Enable TLS and assume server role during TLS handshake 
ca ca.crt         # Certificate authority (CA) file
dh dh1024.pem     # File containing Diffie Hellman parameters 
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
cert server.crt   # Local peer's signed certificate
key server.key    # Local peer's private key 

and config 2:
openvpntcp.conf

Code: Select all

# Tunnel options
mode server       # Set OpenVPN major mode
proto tcp-server        # Setup the protocol (server)
port 1195         # TCP/UDP port number
dev tap          # TUN/TAP virtual network device
keepalive 15 60   # Simplify the expression of --ping 
daemon            # Become a daemon after all initialization
verb 3            # Set output verbosity to n 
#comp-lzo          # Use fast LZO compression

# OpenVPN server mode options
client-to-client  # tells OpenVPN to internally route client-to-client traffic 
duplicate-cn      # Allow multiple clients with the same common name

# TLS Mode Options
tls-server        # Enable TLS and assume server role during TLS handshake 
ca ca.crt         # Certificate authority (CA) file
dh dh1024.pem     # File containing Diffie Hellman parameters 
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
cert server.crt   # Local peer's signed certificate
key server.key    # Local peer's private key 

Firewall:

Code: Select all

insmod ipt_mark
insmod xt_mark
iptables -t mangle -A PREROUTING -i ! `get_wanface` -d `nvram get wan_ipaddr` -j MARK --set-mark 0xd001
iptables -t mangle -A PREROUTING -j CONNMARK --save-mark
iptables -t nat -A POSTROUTING -m mark --mark 0xd001 -j MASQUERADE
# Open firewall holes
iptables -I INPUT 2 -p tcp --dport 1195 -j ACCEPT
iptables -I FORWARD -i br0 -o tun0 -j ACCEPT
iptables -I FORWARD -i tun0 -o br0 -j ACCEPT

Clients:

the clients which use UDP are working:

sleep 5
ln -s /usr/sbin/openvpn /tmp/myvpn

/tmp/myvpn --mktun --dev tap0
brctl addif br0 tap0
ifconfig tap0 0.0.0.0 promisc up
sleep 5
/tmp/myvpn --config /tmp/client.conf
route add -net 192.168.1.0/24 dev br0
sleep 15 ; insmod /lib/modules/2.4.37/ebtables ; insmod /lib/modules/2.4.37/ebtable_filter ; insmod /lib/modules/2.4.37/ebt_ip.o &
sleep 25 ; /usr/sbin/ebtables -I INPUT -i tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &
sleep 5 ; /usr/sbin/ebtables -I OUTPUT -o tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &

client.conf

Code: Select all

client
daemon
float
dev tap0
proto udp
remote barney.ro 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca /tmp/ca.crt
cert /tmp/client.crt
key /tmp/client.key
ns-cert-type server
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
#comp-lzo adaptive
verb 3

The client config with tcp (connects but cannot ping the rest of the network):

Code: Select all

 Automatically generated configuration 
daemon 
client 
dev tap11 
proto udp 
remote barney.ro 1194 
resolv-retry infinite 
nobind 
persist-key 
persist-tun 
cipher none 
verb 3 
script-security 2 
up updown.sh 
down updown.sh 
ca ca.crt 
cert client.crt 
key client.key 
status-version 2 
status status 
 
# Custom Configuration 
float 
ns-cert-type server 
tls-cipher AES256-SHA 

Any help would be appreciated.

[trex_daemon]

Ok, I've studied a lot the problem, and did some corrections.
I will write down each step so that you can correct me if my understanding is wrong or I did not do something correctly.

On the server side:
1) create the TUN virtual network devices
- used by vpn client running on UDP 1194

Code: Select all

openvpn --mktun --dev tap0

- used by vpn client running on TCP 1195

Code: Select all

openvpn --mktun --dev tap1

2) add the 2 interfaces to create a bridge between the other networks.

Code: Select all

brctl addif br0 tap0
brctl addif br0 tap1

At this point, if I issue a

Code: Select all

brctl show

command, I get the following output:

Code: Select all

root@trex:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.001d7e60a647       no              vlan0
                                                        eth1
                                                        tap0
                                                        tap1

Then I issue the following 2 commands, to bring both TAP interfaces into promiscous mode:

[-]promisc
Enable or disable the promiscuous mode of the interface. If selected, all packets on the network will be
received by the interface.

Code: Select all

ifconfig tap0 0.0.0.0 promisc up
ifconfig tap1 0.0.0.0 promisc up

Then I create a symbolic link to the openvpn and start the 2 VPN servers. One is listening on UDP 1194, the other one on TCP 1195.

Code: Select all

ln -s /usr/sbin/openvpn /tmp/myvpn
/tmp/myvpn --config openvpn.conf
/tmp/myvpn --config openvpntcp.conf

the 2 config files are:
for UDP 1194: openvpn.conf

Code: Select all

# Tunnel options
mode server       # Set OpenVPN major mode
proto udp         # Setup the protocol (server)
port 1194         # TCP/UDP port number
dev tap0          # TUN/TAP virtual network device
keepalive 15 60   # Simplify the expression of --ping 
daemon            # Become a daemon after all initialization
verb 3            # Set output verbosity to n 
#comp-lzo          # Use fast LZO compression

# OpenVPN server mode options
client-to-client  # tells OpenVPN to internally route client-to-client traffic 
duplicate-cn      # Allow multiple clients with the same common name

# TLS Mode Options
tls-server        # Enable TLS and assume server role during TLS handshake 
ca ca.crt         # Certificate authority (CA) file
dh dh1024.pem     # File containing Diffie Hellman parameters 
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
cert server.crt   # Local peer's signed certificate
key server.key    # Local peer's private key 

for TCP 1195: openvpntcp.conf:

Code: Select all

# Tunnel options
mode server       # Set OpenVPN major mode
proto tcp-server        # Setup the protocol (server)
port 1195         # TCP/UDP port number
dev tap1          # TUN/TAP virtual network device
keepalive 15 60   # Simplify the expression of --ping 
daemon            # Become a daemon after all initialization
verb 3            # Set output verbosity to n 
#comp-lzo          # Use fast LZO compression

# OpenVPN server mode options
client-to-client  # tells OpenVPN to internally route client-to-client traffic 
duplicate-cn      # Allow multiple clients with the same common name

# TLS Mode Options
tls-server        # Enable TLS and assume server role during TLS handshake 
ca ca.crt         # Certificate authority (CA) file
dh dh1024.pem     # File containing Diffie Hellman parameters 
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
cert server.crt   # Local peer's signed certificate
key server.key    # Local peer's private key 

In the first config I've used device tap0 , in the second config device tap1.

After starting the 2 openvpn server, I'm issuing the following command:

Code: Select all

route add -net 192.168.1.0/24 dev br0

This should route all traffic from 192.168.1.0/24 to bridg[quote][/quote]e device 0.

All these commands are issued from a startup script from a router running ddwrt.

I'm using several routers which are connected to the network. The ip addresses from the routers are 192.168.1.1 (the server) and the rest of them are 192.168.1.254 -> 192.168.1.250.
DCHP is configured to provide different address ranges, so the IP's won't overlap.

on the client routers ebtables are also used to drop DHCP requests from the other networks.

THe 2 ports on the firewall are also open:

Code: Select all

# Open firewall holes
iptables -I INPUT 2 -p tcp --dport 1195 -j ACCEPT
iptables -I INPUT 2 -p udp --dport 1194 -j ACCEPT

Port forwarding is also set up

Code: Select all

iptables -I FORWARD -i br0 -o tun0 -j ACCEPT
iptables -I FORWARD -i tun0 -o br0 -j ACCEPT
iptables -I FORWARD -i br0 -o tun1 -j ACCEPT
iptables -I FORWARD -i tun1 -o br0 -j ACCEPT

Now the clients:
This part is common, and already describe above:

Code: Select all

ln -s /usr/sbin/openvpn /tmp/myvpn
/tmp/myvpn --mktun --dev tap0
brctl addif br0 tap0
ifconfig tap0 0.0.0.0 promisc up

Then the VPN server is started.

Code: Select all

/tmp/myvpn --config /tmp/client.conf

Routing table updated:

Code: Select all

route add -net 192.168.1.0/24 dev br0

And here's the client.conf:

Code: Select all

client
daemon
float
dev tap0
proto udp
remote barney.ro 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca /tmp/ca.crt
cert /tmp/client.crt
key /tmp/client.key
ns-cert-type server
#cipher AES-192-CBC
cipher none
tls-cipher AES256-SHA
#comp-lzo adaptive
verb 3

Ebtables is also configured:

Code: Select all

sleep 25 ; /usr/sbin/ebtables -I INPUT -i tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP & 
sleep 5 ; /usr/sbin/ebtables -I OUTPUT -o tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &

Clients which use UPD do work. They see each other, also the router sees them and can ping them.

Now the client that is connecting via TCP , although it connects, I cannot see the rest of the network.
Here are the steps:

Code: Select all

/usr/sbin/openvpn --mktun --dev tap12
brctl addif br0 tap12
ifconfig tap12 0.0.0.0 promisc up
sleep 1
route add -net 192.168.1.0/24 dev br0

EBTABLES:

Code: Select all

/usr/sbin/ebtables -I INPUT -i tap12 -p IPv4 --ip-protocol tcp --ip-destination-port 67:68 -j DROP & 
/usr/sbin/ebtables -I OUTPUT -o tap12 -p IPv4 --ip-protocol tcp --ip-destination-port 67:68 -j DROP &

How does this work ? Shall I use here TCP or UDP ?


The client configuration file:

Code: Select all

# Automatically generated configuration 
daemon 
client 
dev tap12 
proto tcp-client 
remote barney.ro 1195 
resolv-retry infinite 
nobind 
persist-key 
persist-tun 
comp-lzo adaptive 
cipher none 
verb 3 
ca ca.crt 
cert client.crt 
key client.key 
status-version 2 
status status 
 
# Custom Configuration 
tls-cipher AES256-SHA 
ns-cert-type server 
float

Here's the traceroute issued from this router:

Code: Select all

traceroute to 192.168.1.3 (192.168.1.3), 30 hops max, 38 byte packets
 1  *  *  *
 2  *  *  budapest (192.168.1.254)  4960.120 ms !H
 3  *  *  *
 4

So, any help is appreciated again

[trex_daemon]

I've found the root cause:
The

Code: Select all

comp-lzo adaptive

was active

Now it's working.
The thread can be closed. However I think this might be a great tutorial to somebody who's struggling with the same problems as myself.

Code: Select all

# Automatically generated configuration
daemon
client
dev tap12
proto tcp-client
remote barney.ro 1195
resolv-retry infinite
nobind
persist-key
persist-tun
[color=#FF4000]#comp-lzo adaptive[/color]
cipher none
verb 3
ca ca.crt
cert client.crt
key client.key
status-version 2
status status
 
# Custom Configuration
tls-cipher AES256-SHA
ns-cert-type server
float

[trex_daemon]

Just for the record, I would like to put here also the correct settings for ebtables:

#optional to load modules
sleep 1 ; insmod /lib/modules/2.4.37/ebtables ; insmod /lib/modules/2.4.37/ebtable_filter ; insmod /lib/modules/2.4.37/ebt_ip.o &


sleep 1 ; /usr/sbin/ebtables -I INPUT -i tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &
sleep 1 ; /usr/sbin/ebtables -I OUTPUT -o tap0 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &
sleep 1 ; /usr/sbin/ebtables -I INPUT -i tap1 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &
sleep 1 ; /usr/sbin/ebtables -I OUTPUT -o tap1 -p IPv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP &

where tap0 and tap 1 are the virtual network devices.
These lines will block the DHCP requests that are sent through the virtual netowork devices.

[Pariah]

I happened to see this looking for answers to my issue, BUT I think I can help you further simplify this.

For DHCP or auto address assign you should just set your DHCP server to only pull from a pool of ips that exclude those used by any VPN clients [like if using 192.168.1.0/24 tell DHCP server to only assign ip 02 [most routers use 01 as default] to say 200 right. Then in your VPN SERVER config use the following directive

server-bridge 192.168.1.4 255.255.255.0 192.168.1.200 192.168.1.254
definitions: ^a bridge ^server IP is ^subnet is ^First CLIENT IP ^last CLIENT IP.

Then set your router's DHCP server IP pool to 192.168.1.2 - 192.168.1.199 [Even my ISP supplied router has this in advanced DHCP server settings].

done and done AND no need to ever worry about filtering DHCP packets AND no extra packets heading over VPN link to ignore and waste bandwidth!

Then as clients connects you can either have it auto choose a number or have persistence turned on so client gets same IP OR set a client config to assign a desired IP to a desired Client. I just set persistence on [persist key and tun (optional)] and let the OpenVPN serve the address it wishes. Usually same client will retain same address unless another client using same cert connects and it changes [mostly no big deal as I have DNS and all so I personally don't care what IP they get]. ANYWAYS, this way you do not have your DHCP server confused [if it is not set to ignore the IPs OpenVPN uses and it tries to assign the same IP a VPN client has to a non VPN client at the same time one of them will have no network access]. Also, no need to worry about filtering DHCP packets that's overkill and a waste of both resources and bandwidth [even though client ignores packets they are still sent and resent].

See this page as it describes exactly this: http://openvpn.net/index.php/open-sourc ... dging.html . Only thing it may not mention is go in to your DHCP server [my ISP router is capable of this you just have to look for advanced DHCP settings] and limit the IPs it has in it's pool to assign and viola done.

This works as OpenVPN will assign the address and appear as a DHCP server to the client and client will state it connected to DHCP in logs but this is the openvpn server and it will then handle everything for you.

[Pariah]

Actually, as I read it on the howto page it specifies a LAN IP not gateway IP in the directions meaning to me the IP of the server since in the example it is .4 and not the usual default of .1 used to usually specify the root gateway of a network in a howto [http://openvpn.net/index.php/open-sourc ... dging.html]

In fact this is how my VPN is setup, and makes sense from a logical point of view to myself. Once anything is set as a bridge if you aim a gateway directive at it it should be the same thing as the bridge is a gateway from one network subnet to another. I am sure the root gateway would work as well in a bridged environ as the whole subnet is exposed, but technically to the VPN clients the VPN server is their gateway to the rest of the network.

Bridged setups rock, without it my Media streaming server Plex would not work [it uses subnet broadcasts]. With a bridged VPN it works just as it does at home .

[trex_daemon]

Hello,
Sorry for the late late answer,
I got my setup working and did not check this thread any more.
Thanks also for the suggestions!
Indeed, this way the ebtables can be removed completely, which is good, as the tomato builds offer no ebtables with 2.4 kernels.
And I really like the tomato firwares.
Yes, plex was also one of the reasons why I wanted TAP instead of TUN.