ethernet tunnel (not bridge) random PING problem

[enjoyjoy]

Hi There is a challenge and strange problem that need your help.

===== Server Configuration ==========
dev tap
proto udp
server 10.88.0.0 255.255.0.0
client-to-client

The openvpn server is on the CentOS 5 and client computers run Windows XP with firewall turned off.

There are about 400+ client.
Most of the time, clients to client communication is OK, but there are sometime and situations, some of the clients can't contact. Let's suppose there are two clients are named Client A , Client B and the openvpn server is named S.


1) Ping from S to A , failed
2) Ping from A to S, with one ping icmp packet timeout, the second and all followings success.
AS SOON AS A->S success, ping from S to A will be OK from that time on.

3) Ping from A to B (whether before or after A ping S successfully) always fails.

Tcpdump shows that the ARP packet already arrived at openvpn server S, but without return the MAC address of B.
Even add the static MAC address of B on A, A can't ping B successfully.

We are sure there is no firewall/iptable mis configuration in the client computers or openvpn server.

Any idea for further troubleshooting. Your time is of great appreciated!

[enjoyjoy]

This happens only when there are 200+ simultaneous clients, but the openvpn server's CPU, memory, network is below half of the server's full capability.

[enjoyjoy]

Any idea,hint, suggestion on how to troubleshooting is of great appreciated!

Is there any company that provides commercial support service on openvpn?

[enjoyjoy]

We are not sure whether it's caused by an incorrect MTU setting.

We'll try to change the mtu setting and verify this.

[krzee]

This happens only when there are 200+ simultaneous clients, but the openvpn server's CPU, memory, network is below half of the server's full capability.

Sorry, I did not see you had said this (i did not even see the thread... i usually scan by seeing ones that have not been answered, or that i have answered... just caught yours via the IRC bot)

please see my post here:
viewtopic.php?p=7923#p7923

[enjoyjoy]

Dear Mr. Krzee,

Thanks for your reply.
Does this means the most recently stable version of openvpn doesn't support 200+ simultaneous clients correctly? I don't find any official document about this limitation.

Thus, we have to run multiple openvpn instances on the single linux machine.
We plan to run two TAP mode openvpn instances and bridge the tow TAP ethernets.
Is this possible?

[krzee]

i do not run tap/bridge setups, so im not sure... but it sounds doable to me
either way, you do NOT want a bridge... 200 or ESPECIALLY 400 machines on a single broadcast domain will cause broadcast storms

and yes, you will need at least 2 server instances... if you run openvpn on a 4 core machine, you could run 4 instances of openvpn without any problem.

I have also not found official documentation about this limitation, but we have discussed it in the developer channel, it does exist.

[enjoyjoy]

Thanks very much for your prompt response.

If this limitation exists in reality, how to create a reliable openvpn network with more than 200 clients, or even more 400 clients?

Does this means tap mode is not recommend if clients is more than 200 while TUN/routing is preferred?

If we run 4 openvpn instances in a single linux machine with 4 TAP subnet(each 100 clients, broadcast), how the 4 subnets communicate with each other? Does bridging the 4 TAP virtual ethernet works?

[krzee]

When you use a tap bridge you extend your LAN. When you extend your LAN to hundreds of clients, you will have problems
This is what was meant when the FAQ says "does not scale well"
http://openvpn.net/index.php/open-sourc ... uting.html

You can probably find something worth reading to understand it better here:
http://www.google.com/search?q=broadcast+domain+too+big

What you need to do is correctly setup routing between the different tun subnets

[enjoyjoy]

The openvpn server is in mode "dev TAP", not using bridge.
There are one TAP adapter in the openvpn server and 200+ simultaneous clients all connect to that TAP.

I uses wireshark to capture all packets in the TAP adapter on one of openvpn clients .

There are netbios broadcast packets originating from the the openvpn client, but there are not any broadcast packets arrive at this tap adapter.

So there is no broadcast storm, right?

[krzee]

personally i dont use bridge so i cant sniff my traffic and give you the right answer

but it is common-sense in networking that you dont want a broadcast domain that big